Jul 24 17:08:12.954861 ip-10-0-140-5 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 17:08:12.954875 ip-10-0-140-5 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 17:08:12.954884 ip-10-0-140-5 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 17:08:12.955352 ip-10-0-140-5 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 17:08:22.817804 ip-10-0-140-5 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 24 17:08:22.817818 ip-10-0-140-5 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b8d27240f321410a8a13d406e7df893f --
Jul 24 17:10:10.915316 ip-10-0-140-5 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 17:10:11.296088 ip-10-0-140-5 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:10:11.296088 ip-10-0-140-5 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 17:10:11.296088 ip-10-0-140-5 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:10:11.296088 ip-10-0-140-5 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 17:10:11.296088 ip-10-0-140-5 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:10:11.299125 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.298952    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 17:10:11.304204 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304157    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304260    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304267    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304272    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304275    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304279    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304281    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304284    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304287    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304290    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304294    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304296    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304299    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304302    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304305    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304307    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304310    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304313    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304316    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:11.304319 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304318    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304321    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304323    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304326    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304329    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304332    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304335    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304338    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304341    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304343    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304346    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304349    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304351    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304354    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304357    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304360    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304362    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304372    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304374    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304377    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:11.304805 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304379    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304382    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304385    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304388    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304390    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304394    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304396    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304402    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304406    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304409    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304412    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304416    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304420    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304423    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304426    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304428    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304431    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304434    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304436    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:11.305340 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304441    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304444    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304448    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304451    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304453    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304456    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304459    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304462    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304465    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304467    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304470    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304472    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304475    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304478    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304481    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304483    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304486    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304489    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304492    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304495    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:11.305804 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304498    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304500    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304503    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304507    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304511    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304515    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304518    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304521    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304524    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304527    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304529    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304532    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304535    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304538    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304541    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304543    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304546    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304548    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304551    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:11.306304 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304553    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.304556    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305327    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305335    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305339    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305341    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305345    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305349    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305354    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305357    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305360    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305363    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305366    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305369    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305372    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305374    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305377    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305379    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305382    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305385    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:11.306779 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305388    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305391    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305394    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305396    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305399    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305401    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305404    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305406    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305409    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305412    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305416    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305419    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305421    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305424    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305426    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305429    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305431    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305434    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305436    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:11.307296 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305440    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305443    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305446    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305449    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305451    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305454    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305457    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305459    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305461    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305464    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305467    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305469    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305483    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305488    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305491    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305494    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305497    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305500    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305502    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305505    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:11.307771 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305508    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305511    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305513    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305516    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305518    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305521    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305523    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305525    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305528    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305530    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305533    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305535    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305538    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305541    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305543    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305545    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305549    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305552    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305555    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305557    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:11.308307 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305560    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305562    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305565    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305567    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305570    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305574    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305577    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305580    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305582    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305585    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305588    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305590    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305592    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305595    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305597    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305600    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305603    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305605    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305608    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305610    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:11.308792 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305612    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.305615    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306037    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306042    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306045    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306048    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306050    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306053    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306056    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306058    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306061    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306063    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306066    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306069    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306071    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306074    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306076    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306079    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306083    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306085    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:11.309288 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306088    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306091    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306093    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306096    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306098    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306101    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306104    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306106    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306108    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306111    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306113    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306116    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306119    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306121    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306124    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306126    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306129    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306131    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306133    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:11.309777 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306136    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306139    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306142    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306144    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306146    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306149    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306151    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306153    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306156    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306158    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306161    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306164    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306172    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306174    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306176    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306179    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306182    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306184    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306187    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306189    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306191    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:11.310344 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306195    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306197    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306200    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306202    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306204    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306207    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306209    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306212    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306216    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306235    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306240    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306244    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306246    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306249    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306253    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306257    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306260    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306262    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:11.310853 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306265    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306267    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306270    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306273    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306275    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306279    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306282    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306284    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306287    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306290    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306292    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306295    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306297    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306300    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306302    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306305    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306307    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306310    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306313    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306315    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:11.311320 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306318    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306320    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.306323    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306393    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306401    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306409    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306413    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306417    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306420    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306425    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306429    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306432    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306437    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306441    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306444    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306447    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306450    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306453    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306456    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306459    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306463    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306467    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306469    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306472    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 17:10:11.311807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306475    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306479    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306482    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306485    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306488    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306491    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306494    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306497    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306500    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306503    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306506    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306510    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306513    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306516    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306519    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306522    2562 flags.go:64] FLAG: --enable-server="true"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306525    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306530    2562 flags.go:64] FLAG: --event-burst="100"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306533    2562 flags.go:64] FLAG: --event-qps="50"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306536    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306540    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306543    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306546    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306549    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306552    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 17:10:11.312408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306555    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306558    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306560    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306564    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306567    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306570    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306572    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306575    2562 flags.go:64] FLAG: --feature-gates=""
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306579    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306582    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306585    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306588    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306591    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306593    2562 flags.go:64] FLAG: --help="false"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306596    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306599    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306602    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306605    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306608    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306611    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306614    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306617    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306620    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 17:10:11.313010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306623    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306626    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306629    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306631    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306636    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306638    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306641    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306644    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306647    2562 flags.go:64] FLAG: --lock-file=""
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306649    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306652    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306655    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306660    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306664    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306667    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306670    2562 flags.go:64] FLAG: --logging-format="text"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306673    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306676    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306678    2562 flags.go:64] FLAG: --manifest-url=""
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306681    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306685    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306688    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306692    2562 flags.go:64] FLAG: --max-pods="110"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306695    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306698    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 17:10:11.313684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306700    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306703    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306706    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306709    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306712    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306719    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306722    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306725    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306728    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306731    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306736    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306740    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306743    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306746    2562 flags.go:64] FLAG: --port="10250"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306749    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306751    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-009bde57a96043c75"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306755    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306757    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306760    2562 flags.go:64] FLAG: --register-node="true"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306763    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306767    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306770    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306773    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306776    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306779    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 17:10:11.314299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306783    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306786    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306789    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306792    2562 flags.go:64] FLAG: --runonce="false"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306795    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306798    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306800    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306803    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306806    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306809    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306812    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306815    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306817    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306820    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306824    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306827    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306830    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306833    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306837    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306842    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306845    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306847    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306851    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306854    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306857    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 17:10:11.314889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306859    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306862    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306865    2562 flags.go:64] FLAG: --v="2"
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306869    2562 flags.go:64] FLAG: --version="false"
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306873    2562 flags.go:64] FLAG: --vmodule=""
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306877    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.306880    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307005    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307010    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307013    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307032    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307035    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307038    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307041    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307043    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307046    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307048    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307051    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307053    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307056    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307058    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:11.315553 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307061    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307063    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307066    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307070    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307075    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307078    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307081    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307083    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307086    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307088    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307091    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307093    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307096    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307098    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307100    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307103    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307106    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307109    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307111    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307113    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:11.316097 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307116    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307118    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307121    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307123    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307126    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307128    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307131    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307134    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307136    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307139    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307142    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307144    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307150    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307152    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307155    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307157    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307161    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307163    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307166    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307168    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:11.316574 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307171    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307174    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307176    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307179    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307181    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307184    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307186    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307189    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307191    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307195    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307197    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307200    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307202    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307205    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307207    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307211    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307215    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307217    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307220    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:11.317170 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307223    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307226    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307228    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307231    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307233    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307237    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307240    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307244    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307247    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307251    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307254    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307256    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307259    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307261    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307264    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307266    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307269    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307271    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307274    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:11.317636 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307276    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307279    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307281    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307284    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307287    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307290    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.307292    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.308218    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.314765    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.314783    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.315515    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.315521    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315579    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315584    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:11.318106 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315588    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315591    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315594    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315596    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315599    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315602    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315605    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315608    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315611    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315614    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315617    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315620    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315623    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315625    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315628    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315630    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315633    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315636    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315638    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315641    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:11.318528 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315643    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315646    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315648    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315651    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315653    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315656    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315658    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315662    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315665    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315669    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315673    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315676    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315687    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315691    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315694    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315696    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315699    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315702    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315704    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:11.319076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315707    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315710    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315712    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315715    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315717    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315720    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315723    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315725    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315729    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315733    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315736    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315738    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315742    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315745    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315748    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315751    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315754    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315756    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315759    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:11.319566 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315761    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315765    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315769    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315772    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315774    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315777    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315780    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315782    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315785    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315787    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315789    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315792    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315794    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315797    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315799    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315802    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315804    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315807    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315809    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315812    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:11.320257 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315814    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315817    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315819    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315822    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315824    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315827    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315829    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315832    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315835    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315837    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315839    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315842    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315844    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315848    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315851    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315857    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315859    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315862    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:11.320925 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315864    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.315869    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315985    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315989    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315992    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315995    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.315998    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316001    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316003    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316006    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316008    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316010    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316013    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316028    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316031    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:10:11.321594 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316034    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316036    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316039    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316041    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316044    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316047    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316050    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316052    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316055    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316073    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316077    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316080    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316084    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316088    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316091    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316094    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316097    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316101    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316104    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:10:11.322076 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316107    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316109    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316112    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316115    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316118    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316120    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316123    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316126    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316128    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316131    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316135    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316138    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316141    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316143    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316146    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316148    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316151    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316153    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316156    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:10:11.322608 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316158    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316160    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316163    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316165    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316168    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316170    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316173    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316175    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316178    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316180    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316184    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316188    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316191    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316193    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316196    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316198    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316200    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316203    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316205    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316208    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:10:11.323348 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316210    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316213    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316215    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316217    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316220    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316223    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316225    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316227    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316230    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316232    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316235    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316237    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316240    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316242    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316245    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316247    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316249    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316252    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316255    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316257    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:10:11.323988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316260    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316262    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316265    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316268    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316270    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316273    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316275    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:11.316277    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.316282    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.316395    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.318333    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.319530    2562 server.go:1019] "Starting client certificate rotation"
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.319646    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.319730    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:10:11.324579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.322340    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:10:11.325058 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.323590    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:10:11.334976 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.334957    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 24 17:10:11.338967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.338951    2562 log.go:25] "Validated CRI v1 image API"
Jul 24 17:10:11.339794 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.339779    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 17:10:11.342670 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.342651    2562 fs.go:135] Filesystem UUIDs: map[66198edd-4012-4b61-97f6-775a53c2d033:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8af60e65-eeb9-4008-93fe-e24768094acd:/dev/nvme0n1p4]
Jul 24 17:10:11.342738 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.342671    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 17:10:11.348447 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.348332    2562 manager.go:216] Machine: {Timestamp:2026-07-24 17:10:11.346302839 +0000 UTC m=+0.317435011 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100414 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b082551e8279d595a4d5d0fdc58d3 SystemUUID:ec2b0825-51e8-279d-595a-4d5d0fdc58d3 BootID:b8d27240-f321-410a-8a13-d406e7df893f Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:52:93:a8:51:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:52:93:a8:51:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:22:e6:1b:05:82 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 17:10:11.348447 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.348436    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 17:10:11.348561 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.348520    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 17:10:11.351642 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.351613    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 17:10:11.351788 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.351643    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-5.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 17:10:11.352292 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.352280    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 17:10:11.352324 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.352295    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 17:10:11.352324 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.352303    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:10:11.352855 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.352833    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:10:11.352949 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.352898    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:10:11.353622 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.353610    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 17:10:11.356278 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356264    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:10:11.356396 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356386    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 17:10:11.356448 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356416    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 17:10:11.356448 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356427    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 17:10:11.356448 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356445    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 17:10:11.356555 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356455    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 24 17:10:11.356555 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356465    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 17:10:11.356937 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356923    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:10:11.356974 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.356952    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:10:11.359004 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.358987    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 17:10:11.360957 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.360941    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 17:10:11.361033 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.360973    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 17:10:11.362460 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362447    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 17:10:11.362508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362464    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 17:10:11.362508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362470    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 17:10:11.362508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362476    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 17:10:11.362508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362484    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 17:10:11.362508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362489    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 17:10:11.362508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362497    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 17:10:11.362508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362502    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 17:10:11.362687 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362514    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 17:10:11.362687 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362519    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 17:10:11.362687 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362531    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 17:10:11.362687 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.362536    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 17:10:11.363846 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.363835    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 17:10:11.363846 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.363847    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 17:10:11.366310 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.366288    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 17:10:11.366473 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.366449    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-5.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 17:10:11.366597 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.366582    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-5.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 17:10:11.367442 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.367429    2562 server.go:1268] "Started kubelet"
Jul 24 17:10:11.367647 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.367589    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 17:10:11.367780 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.367718    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 17:10:11.367780 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.367584    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 17:10:11.371767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.371735    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 17:10:11.371770 ip-10-0-140-5 systemd[1]: Started Kubernetes Kubelet.
Jul 24 17:10:11.374061 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.374036    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9wgsd"
Jul 24 17:10:11.375138 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.375121    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 17:10:11.377611 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.377592    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 17:10:11.379560 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.379547    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 17:10:11.379986 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.379969    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 17:10:11.380197 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.380177    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:11.383057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.380920    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 17:10:11.383057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.380936    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 17:10:11.383057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.381070    2562 factory.go:55] Registering systemd factory
Jul 24 17:10:11.383057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.381084    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 24 17:10:11.383057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.381094    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 17:10:11.383057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.381103    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 17:10:11.383057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.381144    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 17:10:11.383427 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.383063    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9wgsd"
Jul 24 17:10:11.383427 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.383352    2562 factory.go:153] Registering CRI-O factory
Jul 24 17:10:11.383427 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.383370    2562 factory.go:223] Registration of the crio container factory successfully
Jul 24 17:10:11.383659 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.383438    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 17:10:11.383659 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.383463    2562 factory.go:103] Registering Raw factory
Jul 24 17:10:11.383659 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.383478    2562 manager.go:1196] Started watching for new ooms in manager
Jul 24 17:10:11.384145 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.384120    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-5.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 17:10:11.384247 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.384188    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 17:10:11.384305 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.384247    2562 manager.go:349] Starting recovery of all containers
Jul 24 17:10:11.385246 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.384226    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-5.ec2.internal.18c548ed4d8680a8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-5.ec2.internal,UID:ip-10-0-140-5.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-5.ec2.internal,},FirstTimestamp:2026-07-24 17:10:11.367379112 +0000 UTC m=+0.338511284,LastTimestamp:2026-07-24 17:10:11.367379112 +0000 UTC m=+0.338511284,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-5.ec2.internal,}"
Jul 24 17:10:11.394733 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.394617    2562 manager.go:354] Recovery completed
Jul 24 17:10:11.399422 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.399409    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:11.401714 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.401698    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:11.401789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.401725    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:11.401789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.401736    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:11.402302 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.402285    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 17:10:11.402302 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.402300    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 17:10:11.402302 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.402304    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:11.402454 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.402318    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:10:11.402980 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.402966    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:11.403083 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.402992    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:11.403083 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.403001    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:11.404809 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.404796    2562 policy_none.go:49] "None policy: Start"
Jul 24 17:10:11.404851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.404814    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 17:10:11.404851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.404824    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 17:10:11.406755 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.406744    2562 policy_none.go:47] "Start"
Jul 24 17:10:11.453689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.453663    2562 manager.go:337] "Starting Device Plugin manager"
Jul 24 17:10:11.453773 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.453711    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 17:10:11.453773 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.453725    2562 server.go:87] "Starting device plugin registration server"
Jul 24 17:10:11.454065 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.454052    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 17:10:11.454110 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.454067    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 17:10:11.454172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.454155    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 17:10:11.454250 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.454238    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 17:10:11.454295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.454250    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 17:10:11.454843 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.454819    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 17:10:11.454919 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.454881    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:11.497616 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.497584    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 17:10:11.504345 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.498861    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 17:10:11.504345 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.498898    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 17:10:11.504345 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.498919    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 17:10:11.504345 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.498954    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 17:10:11.504345 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.501709    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:11.554917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.554832    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:11.555859 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.555840    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:11.555952 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.555879    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:11.555952 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.555890    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:11.555952 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.555916    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.561668 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.561651    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.561754 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.561678    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-5.ec2.internal\": node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:11.575856 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.575838    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:11.599345 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.599315    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal"]
Jul 24 17:10:11.599442 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.599388    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:11.600400 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.600385    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:11.600475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.600413    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:11.600475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.600423    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:11.602708 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.602696    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:11.602817 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.602804    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.602856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.602831    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:11.603509 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.603492    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:11.603581 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.603524    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:11.603581 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.603538    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:11.603581 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.603502    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:11.603668 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.603587    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:11.603668 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.603597    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:11.605739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.605723    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.605805 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.605754    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:10:11.606428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.606403    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:10:11.606496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.606440    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:10:11.606496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.606450    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:10:11.619710 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.619688    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-5.ec2.internal\" not found" node="ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.623614 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.623599    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-5.ec2.internal\" not found" node="ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.676873 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.676842    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:11.682874 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.682850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8d767b07b5a78a429c99353bd6c317f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal\" (UID: \"8d767b07b5a78a429c99353bd6c317f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.682986 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.682884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d767b07b5a78a429c99353bd6c317f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal\" (UID: \"8d767b07b5a78a429c99353bd6c317f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.682986 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.682909    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c39321f7468a5658b1cdb40ffa446cd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-5.ec2.internal\" (UID: \"9c39321f7468a5658b1cdb40ffa446cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.777306 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.777273    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:11.783702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.783669    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8d767b07b5a78a429c99353bd6c317f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal\" (UID: \"8d767b07b5a78a429c99353bd6c317f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.783809 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.783716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d767b07b5a78a429c99353bd6c317f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal\" (UID: \"8d767b07b5a78a429c99353bd6c317f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.783809 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.783740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c39321f7468a5658b1cdb40ffa446cd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-5.ec2.internal\" (UID: \"9c39321f7468a5658b1cdb40ffa446cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.783809 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.783785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8d767b07b5a78a429c99353bd6c317f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal\" (UID: \"8d767b07b5a78a429c99353bd6c317f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.783934 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.783792    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d767b07b5a78a429c99353bd6c317f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal\" (UID: \"8d767b07b5a78a429c99353bd6c317f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.783934 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.783791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9c39321f7468a5658b1cdb40ffa446cd-config\") pod \"kube-apiserver-proxy-ip-10-0-140-5.ec2.internal\" (UID: \"9c39321f7468a5658b1cdb40ffa446cd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.878194 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.878125    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:11.922629 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.922598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.927395 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:11.927254    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:11.979285 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:11.979250    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:12.079714 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:12.079690    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:12.186189 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:12.186100    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:12.256203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.256174    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:12.286819 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:12.286786    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:12.320494 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.320470    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 17:10:12.321034 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.320629    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:10:12.321034 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.320642    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:10:12.387396 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:12.387365    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:12.388504 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.388477    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 17:05:12 +0000 UTC" deadline="2027-12-27 08:17:44.974874165 +0000 UTC"
Jul 24 17:10:12.388564 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.388505    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12495h7m32.586371976s"
Jul 24 17:10:12.488428 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:12.488344    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:12.558337 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.558304    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:12.589356 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:12.589321    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:12.646052 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.646029    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:10:12.690108 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:12.690073    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-5.ec2.internal\" not found"
Jul 24 17:10:12.780079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.780015    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:12.781065 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.781015    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:12.792577 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.792550    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:10:12.793553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.793539    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal"
Jul 24 17:10:12.799937 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:12.799916    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:10:13.359739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.359706    2562 apiserver.go:52] "Watching apiserver"
Jul 24 17:10:13.374390 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.373803    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 17:10:13.375964 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.375537    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s","openshift-cluster-node-tuning-operator/tuned-fq6nj","openshift-image-registry/node-ca-45lzv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal","openshift-multus/multus-cfbd8","openshift-multus/network-metrics-daemon-8qx5c","openshift-network-diagnostics/network-check-target-th2mm","kube-system/konnectivity-agent-4ngr7","kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal","openshift-dns/node-resolver-bmq8m","openshift-multus/multus-additional-cni-plugins-2lfq6","openshift-network-operator/iptables-alerter-267rb","openshift-ovn-kubernetes/ovnkube-node-zp6nk","kube-system/global-pull-secret-syncer-ptxkr"]
Jul 24 17:10:13.378416 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.378389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.382402 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.381238    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.382402 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.381714    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 17:10:13.383041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.382822    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 17:10:13.383041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.382852    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 17:10:13.383221 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.383195    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-56wmj\""
Jul 24 17:10:13.383932 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.383913    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 17:10:13.384700 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.384146    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 17:10:13.384700 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.384309    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d9fj6\""
Jul 24 17:10:13.384700 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.384431    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 17:10:13.384700 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.384588    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 17:10:13.385108 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.384792    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.385108 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.384872    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 17:10:13.385315 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.385289    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 17:10:13.386795 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.386774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.386885 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.386848    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:13.387335 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.387314    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 17:10:13.387444 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.387404    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 17:10:13.387654 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.387632    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rhhrp\""
Jul 24 17:10:13.388108 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.387796    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 17:10:13.389484 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.389465    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.392353 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.392329    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 17:10:13.392701 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.392686    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 17:10:13.393483 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.392966    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4qgk5\""
Jul 24 17:10:13.393483 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.393287    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 17:10:13.393483 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.393481    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 17:10:13.394277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394257    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:10:13.394608 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394573    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-run-ovn-kubernetes\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394682 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394607    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-cni-netd\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394682 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394632    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-etc-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394682 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-ovn\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-cni-bin\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-env-overrides\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-ovnkube-script-lib\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-serviceca\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.394808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394792    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cxt7\" (UniqueName: \"kubernetes.io/projected/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-kube-api-access-7cxt7\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.394982 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wwmg\" (UniqueName: \"kubernetes.io/projected/c8135909-83dd-4abb-9121-2c9b7dbd77ec-kube-api-access-4wwmg\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.394982 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-node-log\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394982 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394891    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394982 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/697f9852-7423-45f4-851e-89da6d7e593f-ovn-node-metrics-cert\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.394982 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394968    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-systemd-units\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.394993    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-slash\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395017    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-log-socket\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-var-lib-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395112    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8135909-83dd-4abb-9121-2c9b7dbd77ec-hosts-file\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.395209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395139    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-run-netns\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395164    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-systemd\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395188    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-ovnkube-config\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395452 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395212    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-host\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.395452 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8135909-83dd-4abb-9121-2c9b7dbd77ec-tmp-dir\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.395452 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw6k6\" (UniqueName: \"kubernetes.io/projected/697f9852-7423-45f4-851e-89da6d7e593f-kube-api-access-nw6k6\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395452 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395287    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395452 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395318    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-kubelet\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.395826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.395806    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:13.395908 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.395876    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:13.396369 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.396186    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:13.398996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.398617    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:13.398996 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.398679    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:13.398996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.398700    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 17:10:13.398996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.398804    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 17:10:13.398996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.398926    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zsq7g\""
Jul 24 17:10:13.402383 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.402356    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.405379 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.404954    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:10:13.405379 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.404985    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 17:10:13.405379 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.405106    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.405649 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.405630    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-25h47\""
Jul 24 17:10:13.408267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.408058    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dclm7\""
Jul 24 17:10:13.408267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.408162    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 17:10:13.408267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.408216    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 17:10:13.408472 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.408384    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 17:10:13.409692 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.409309    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.413027 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.411697    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.413027 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.412054    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-krf2t\""
Jul 24 17:10:13.413027 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.412300    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 17:10:13.413027 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.412338    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 17:10:13.414295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.414278    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 17:10:13.415431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.415356    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:10:13.416198 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.415887    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-svz4n\""
Jul 24 17:10:13.416198 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.415964    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 17:10:13.421966 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.421941    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x79k8"
Jul 24 17:10:13.427221 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.427190    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x79k8"
Jul 24 17:10:13.483202 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.483173    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 17:10:13.495690 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495656    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-conf-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.495875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495704    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-etc-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.495875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495737    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-cni-bin\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.495875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495767    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-env-overrides\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.495875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-serviceca\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.495875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495822    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-modprobe-d\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.495875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-daemon-config\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.496139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-tuned\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.496139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.496139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495927    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-run\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.496139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-var-lib-kubelet\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.496139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.495976    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-host\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.496139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496039    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-cnibin\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.496139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-os-release\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.496139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k79cc\" (UniqueName: \"kubernetes.io/projected/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-kube-api-access-k79cc\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496144    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496171    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-cni-bin\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496195    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-etc-kubernetes\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496221    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496252    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/697f9852-7423-45f4-851e-89da6d7e593f-ovn-node-metrics-cert\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-sys\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsgjf\" (UniqueName: \"kubernetes.io/projected/765aa388-ed15-45fd-9f65-7d6204598947-kube-api-access-qsgjf\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496400    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-kubelet-config\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496438    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-var-lib-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-device-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.496513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8135909-83dd-4abb-9121-2c9b7dbd77ec-hosts-file\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496517    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55f4b489-e97a-4542-914e-f06da6c9e49f-cni-binary-copy\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-etc-selinux\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8135909-83dd-4abb-9121-2c9b7dbd77ec-tmp-dir\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496612    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-cni-multus\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496638    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvt4p\" (UniqueName: \"kubernetes.io/projected/55f4b489-e97a-4542-914e-f06da6c9e49f-kube-api-access-kvt4p\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwk4n\" (UniqueName: \"kubernetes.io/projected/573221db-ed4c-48f8-8267-b5eeab52090a-kube-api-access-hwk4n\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-dbus\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-cni-netd\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-cni-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/625c2cb7-a1b2-49c6-800f-91336caf9d82-tmp\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496779    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-system-cni-dir\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-ovn\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496844    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-ovnkube-script-lib\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-socket-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxvr4\" (UniqueName: \"kubernetes.io/projected/d240ac75-8247-4228-bcb0-c5f2c09c882f-kube-api-access-jxvr4\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.496988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-iptables-alerter-script\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cxt7\" (UniqueName: \"kubernetes.io/projected/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-kube-api-access-7cxt7\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.496984    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-cnibin\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-sys-fs\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wwmg\" (UniqueName: \"kubernetes.io/projected/c8135909-83dd-4abb-9121-2c9b7dbd77ec-kube-api-access-4wwmg\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-node-log\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497128    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6769c8c9-a903-4966-9eab-351a6ec580fc-agent-certs\") pod \"konnectivity-agent-4ngr7\" (UID: \"6769c8c9-a903-4966-9eab-351a6ec580fc\") " pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-etc-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497249    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-cni-netd\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-cni-bin\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-node-log\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-env-overrides\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-serviceca\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-var-lib-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497855    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-systemd-units\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.497898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497860    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-ovn\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497883    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-slash\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497911    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-log-socket\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-os-release\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497911    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8135909-83dd-4abb-9121-2c9b7dbd77ec-hosts-file\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.497981    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-log-socket\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498083    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498296    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8135909-83dd-4abb-9121-2c9b7dbd77ec-tmp-dir\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-slash\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498370    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-systemd-units\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-k8s-cni-cncf-io\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-ovnkube-script-lib\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498456    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-kubelet\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-kubernetes\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6769c8c9-a903-4966-9eab-351a6ec580fc-konnectivity-ca\") pod \"konnectivity-agent-4ngr7\" (UID: \"6769c8c9-a903-4966-9eab-351a6ec580fc\") " pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chl4m\" (UniqueName: \"kubernetes.io/projected/625c2cb7-a1b2-49c6-800f-91336caf9d82-kube-api-access-chl4m\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.498767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498591    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-registration-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498636    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysctl-conf\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498718    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-run-netns\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498769    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-run-netns\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-systemd\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498817    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-systemd\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498827    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-ovnkube-config\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-host\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.498994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-host\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499004    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-socket-dir-parent\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499094    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-systemd\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499130    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw6k6\" (UniqueName: \"kubernetes.io/projected/697f9852-7423-45f4-851e-89da6d7e593f-kube-api-access-nw6k6\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499178    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-system-cni-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-netns\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499281    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-hostroot\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-lib-modules\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499355    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-cni-binary-copy\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.499617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499367    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/697f9852-7423-45f4-851e-89da6d7e593f-ovnkube-config\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499417    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-host-slash\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-multus-certs\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499493    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysconfig\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499516    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysctl-d\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-kubelet\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-run-ovn-kubernetes\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-run-openvswitch\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499706    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-kubelet\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.500439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.499731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/697f9852-7423-45f4-851e-89da6d7e593f-host-run-ovn-kubernetes\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.504978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.504893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/697f9852-7423-45f4-851e-89da6d7e593f-ovn-node-metrics-cert\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.506507 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.506373    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal" event={"ID":"9c39321f7468a5658b1cdb40ffa446cd","Type":"ContainerStarted","Data":"8c2fc1469374f25eb84274e4f933f23e6abd46f78c1039d2c455df483142370e"}
Jul 24 17:10:13.510105 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.510009    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal" event={"ID":"8d767b07b5a78a429c99353bd6c317f9","Type":"ContainerStarted","Data":"d6a84b2fad395f6555ce2d2e45b7a82e63455a3aa2488f3fddd5c7a41067ca24"}
Jul 24 17:10:13.511416 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.511390    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wwmg\" (UniqueName: \"kubernetes.io/projected/c8135909-83dd-4abb-9121-2c9b7dbd77ec-kube-api-access-4wwmg\") pod \"node-resolver-bmq8m\" (UID: \"c8135909-83dd-4abb-9121-2c9b7dbd77ec\") " pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.511942 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.511919    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw6k6\" (UniqueName: \"kubernetes.io/projected/697f9852-7423-45f4-851e-89da6d7e593f-kube-api-access-nw6k6\") pod \"ovnkube-node-zp6nk\" (UID: \"697f9852-7423-45f4-851e-89da6d7e593f\") " pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.513057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.513012    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cxt7\" (UniqueName: \"kubernetes.io/projected/8ab674fe-1d10-41b0-90ce-eb0ddfae3042-kube-api-access-7cxt7\") pod \"node-ca-45lzv\" (UID: \"8ab674fe-1d10-41b0-90ce-eb0ddfae3042\") " pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.600235 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6769c8c9-a903-4966-9eab-351a6ec580fc-agent-certs\") pod \"konnectivity-agent-4ngr7\" (UID: \"6769c8c9-a903-4966-9eab-351a6ec580fc\") " pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:13.600235 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600238    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-os-release\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.600431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-k8s-cni-cncf-io\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.600431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-kubelet\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.600431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-kubernetes\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600430    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6769c8c9-a903-4966-9eab-351a6ec580fc-konnectivity-ca\") pod \"konnectivity-agent-4ngr7\" (UID: \"6769c8c9-a903-4966-9eab-351a6ec580fc\") " pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600457    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chl4m\" (UniqueName: \"kubernetes.io/projected/625c2cb7-a1b2-49c6-800f-91336caf9d82-kube-api-access-chl4m\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-k8s-cni-cncf-io\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600481    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-registration-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600502    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-kubelet\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysctl-conf\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-kubernetes\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600549    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-registration-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-socket-dir-parent\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.600604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-systemd\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600630    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-system-cni-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600634    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-os-release\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-socket-dir-parent\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600676    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-netns\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600683    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysctl-conf\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600689    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-systemd\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-netns\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-system-cni-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-hostroot\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-lib-modules\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-cni-binary-copy\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-hostroot\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-host-slash\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-multus-certs\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600949    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysconfig\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600958    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-lib-modules\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.600991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysctl-d\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-conf-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601025    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-run-multus-certs\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601110    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6769c8c9-a903-4966-9eab-351a6ec580fc-konnectivity-ca\") pod \"konnectivity-agent-4ngr7\" (UID: \"6769c8c9-a903-4966-9eab-351a6ec580fc\") " pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-host-slash\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601251    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysconfig\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601258    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-conf-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-sysctl-d\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601375    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-cni-binary-copy\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601363    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-modprobe-d\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-daemon-config\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-tuned\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601457    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601470    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-modprobe-d\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601475    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-run\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-var-lib-kubelet\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-host\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-run\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.601978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601518    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-cnibin\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-var-lib-kubelet\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-cnibin\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601630    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-host\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601610    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-os-release\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601663    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-os-release\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601688    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k79cc\" (UniqueName: \"kubernetes.io/projected/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-kube-api-access-k79cc\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601723    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601750    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-cni-bin\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-etc-kubernetes\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601849    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-cni-bin\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601890    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-sys\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsgjf\" (UniqueName: \"kubernetes.io/projected/765aa388-ed15-45fd-9f65-7d6204598947-kube-api-access-qsgjf\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.602887 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-kubelet-config\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.601954    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601968    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-device-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-daemon-config\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.601992    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55f4b489-e97a-4542-914e-f06da6c9e49f-cni-binary-copy\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602018    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-etc-selinux\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.602085    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret podName:8d915ffa-99b0-4ff7-abc6-ca4c0df718bb nodeName:}" failed. No retries permitted until 2026-07-24 17:10:14.102013379 +0000 UTC m=+3.073145562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret") pod "global-pull-secret-syncer-ptxkr" (UID: "8d915ffa-99b0-4ff7-abc6-ca4c0df718bb") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-kubelet-config\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/625c2cb7-a1b2-49c6-800f-91336caf9d82-sys\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602123    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-cni-multus\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvt4p\" (UniqueName: \"kubernetes.io/projected/55f4b489-e97a-4542-914e-f06da6c9e49f-kube-api-access-kvt4p\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwk4n\" (UniqueName: \"kubernetes.io/projected/573221db-ed4c-48f8-8267-b5eeab52090a-kube-api-access-hwk4n\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602201    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-etc-selinux\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-dbus\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602239    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-cni-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602260    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-etc-kubernetes\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/625c2cb7-a1b2-49c6-800f-91336caf9d82-tmp\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.603818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602307    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-system-cni-dir\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-socket-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602346    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-device-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxvr4\" (UniqueName: \"kubernetes.io/projected/d240ac75-8247-4228-bcb0-c5f2c09c882f-kube-api-access-jxvr4\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-iptables-alerter-script\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-cnibin\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-sys-fs\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-dbus\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-host-var-lib-cni-multus\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.602813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-multus-cni-dir\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.603006    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/55f4b489-e97a-4542-914e-f06da6c9e49f-cni-binary-copy\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.603050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.603070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/55f4b489-e97a-4542-914e-f06da6c9e49f-cnibin\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.602425    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.603166    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs podName:573221db-ed4c-48f8-8267-b5eeab52090a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:14.103147735 +0000 UTC m=+3.074279897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs") pod "network-metrics-daemon-8qx5c" (UID: "573221db-ed4c-48f8-8267-b5eeab52090a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:13.604759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.603219    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-sys-fs\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.605702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.603294    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d240ac75-8247-4228-bcb0-c5f2c09c882f-system-cni-dir\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.605702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.603318    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/765aa388-ed15-45fd-9f65-7d6204598947-socket-dir\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.605702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.603404    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-iptables-alerter-script\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.605702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.603639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6769c8c9-a903-4966-9eab-351a6ec580fc-agent-certs\") pod \"konnectivity-agent-4ngr7\" (UID: \"6769c8c9-a903-4966-9eab-351a6ec580fc\") " pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:13.605702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.604333    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d240ac75-8247-4228-bcb0-c5f2c09c882f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.605702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.605300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/625c2cb7-a1b2-49c6-800f-91336caf9d82-tmp\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.608870 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.608844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/625c2cb7-a1b2-49c6-800f-91336caf9d82-etc-tuned\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.613443 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.613381    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:13.613443 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.613403    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:13.613443 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.613414    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hmhtj for pod openshift-network-diagnostics/network-check-target-th2mm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:13.613674 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:13.613490    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj podName:d3a36ac0-84b7-4717-be5b-9bca1c9de683 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:14.113477395 +0000 UTC m=+3.084609560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hmhtj" (UniqueName: "kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj") pod "network-check-target-th2mm" (UID: "d3a36ac0-84b7-4717-be5b-9bca1c9de683") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:13.614368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.614341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chl4m\" (UniqueName: \"kubernetes.io/projected/625c2cb7-a1b2-49c6-800f-91336caf9d82-kube-api-access-chl4m\") pod \"tuned-fq6nj\" (UID: \"625c2cb7-a1b2-49c6-800f-91336caf9d82\") " pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.614597 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.614573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k79cc\" (UniqueName: \"kubernetes.io/projected/a5aa1a5c-d2bf-4780-85ca-730789acc3f3-kube-api-access-k79cc\") pod \"iptables-alerter-267rb\" (UID: \"a5aa1a5c-d2bf-4780-85ca-730789acc3f3\") " pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.615703 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.615678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwk4n\" (UniqueName: \"kubernetes.io/projected/573221db-ed4c-48f8-8267-b5eeab52090a-kube-api-access-hwk4n\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:13.616569 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.616519    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxvr4\" (UniqueName: \"kubernetes.io/projected/d240ac75-8247-4228-bcb0-c5f2c09c882f-kube-api-access-jxvr4\") pod \"multus-additional-cni-plugins-2lfq6\" (UID: \"d240ac75-8247-4228-bcb0-c5f2c09c882f\") " pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.618312 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.618289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvt4p\" (UniqueName: \"kubernetes.io/projected/55f4b489-e97a-4542-914e-f06da6c9e49f-kube-api-access-kvt4p\") pod \"multus-cfbd8\" (UID: \"55f4b489-e97a-4542-914e-f06da6c9e49f\") " pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.618414 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.618369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsgjf\" (UniqueName: \"kubernetes.io/projected/765aa388-ed15-45fd-9f65-7d6204598947-kube-api-access-qsgjf\") pod \"aws-ebs-csi-driver-node-v722s\" (UID: \"765aa388-ed15-45fd-9f65-7d6204598947\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.692241 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.692200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bmq8m"
Jul 24 17:10:13.703035 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.702994    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:13.711277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.711247    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-45lzv"
Jul 24 17:10:13.723309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.723283    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cfbd8"
Jul 24 17:10:13.733280 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.733257    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:13.742201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.742180    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fq6nj"
Jul 24 17:10:13.752243 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.752217    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s"
Jul 24 17:10:13.761211 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.761190    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2lfq6"
Jul 24 17:10:13.777195 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.777171    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-267rb"
Jul 24 17:10:13.842409 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:13.842373    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:14.106865 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:14.106778    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:14.106865 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:14.106822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:14.107119 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:14.106958    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:14.107119 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:14.106959    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:14.107119 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:14.107035    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs podName:573221db-ed4c-48f8-8267-b5eeab52090a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:15.107012891 +0000 UTC m=+4.078145052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs") pod "network-metrics-daemon-8qx5c" (UID: "573221db-ed4c-48f8-8267-b5eeab52090a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:14.107119 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:14.107086    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret podName:8d915ffa-99b0-4ff7-abc6-ca4c0df718bb nodeName:}" failed. No retries permitted until 2026-07-24 17:10:15.107071839 +0000 UTC m=+4.078203999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret") pod "global-pull-secret-syncer-ptxkr" (UID: "8d915ffa-99b0-4ff7-abc6-ca4c0df718bb") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:14.207548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:14.207514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:14.207748 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:14.207671    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:14.207748 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:14.207696    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:14.207748 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:14.207711    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hmhtj for pod openshift-network-diagnostics/network-check-target-th2mm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:14.207899 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:14.207782    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj podName:d3a36ac0-84b7-4717-be5b-9bca1c9de683 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:15.20776164 +0000 UTC m=+4.178893803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmhtj" (UniqueName: "kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj") pod "network-check-target-th2mm" (UID: "d3a36ac0-84b7-4717-be5b-9bca1c9de683") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:14.430382 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:14.430287    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:05:14 +0000 UTC" deadline="2028-05-03 14:29:39.334043924 +0000 UTC"
Jul 24 17:10:14.430382 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:14.430321    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15573h19m24.903728955s"
Jul 24 17:10:14.683020 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:14.682980    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8135909_83dd_4abb_9121_2c9b7dbd77ec.slice/crio-78caa5b088fa5bbfe9ce08a1f7ba5781b4ad6592422d67d5a5e2abfb1dfd9cc1 WatchSource:0}: Error finding container 78caa5b088fa5bbfe9ce08a1f7ba5781b4ad6592422d67d5a5e2abfb1dfd9cc1: Status 404 returned error can't find the container with id 78caa5b088fa5bbfe9ce08a1f7ba5781b4ad6592422d67d5a5e2abfb1dfd9cc1
Jul 24 17:10:14.685088 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:14.684952    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab674fe_1d10_41b0_90ce_eb0ddfae3042.slice/crio-b9edd3446a9c8c7f457d34ecadff6df34c9d09bcd890a71e799fc87d6e74f594 WatchSource:0}: Error finding container b9edd3446a9c8c7f457d34ecadff6df34c9d09bcd890a71e799fc87d6e74f594: Status 404 returned error can't find the container with id b9edd3446a9c8c7f457d34ecadff6df34c9d09bcd890a71e799fc87d6e74f594
Jul 24 17:10:14.698367 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:14.698133    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd240ac75_8247_4228_bcb0_c5f2c09c882f.slice/crio-cb8cb46ffd7e5ce6810f7e85c02f6a0da6cd38e04b202e97061c1e30626e9ae8 WatchSource:0}: Error finding container cb8cb46ffd7e5ce6810f7e85c02f6a0da6cd38e04b202e97061c1e30626e9ae8: Status 404 returned error can't find the container with id cb8cb46ffd7e5ce6810f7e85c02f6a0da6cd38e04b202e97061c1e30626e9ae8
Jul 24 17:10:14.700627 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:14.700597    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod765aa388_ed15_45fd_9f65_7d6204598947.slice/crio-124b9d3c6fd896e933b005b2b8d5abdc4c61d4825c7d372f7b0b5b6c25bddfb8 WatchSource:0}: Error finding container 124b9d3c6fd896e933b005b2b8d5abdc4c61d4825c7d372f7b0b5b6c25bddfb8: Status 404 returned error can't find the container with id 124b9d3c6fd896e933b005b2b8d5abdc4c61d4825c7d372f7b0b5b6c25bddfb8
Jul 24 17:10:14.722751 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:14.722722    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f4b489_e97a_4542_914e_f06da6c9e49f.slice/crio-0670f17f6df626a40f6ce7bc8fa75845e260f58280f160a07e2e2a4599b6857c WatchSource:0}: Error finding container 0670f17f6df626a40f6ce7bc8fa75845e260f58280f160a07e2e2a4599b6857c: Status 404 returned error can't find the container with id 0670f17f6df626a40f6ce7bc8fa75845e260f58280f160a07e2e2a4599b6857c
Jul 24 17:10:14.723481 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:14.723453    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697f9852_7423_45f4_851e_89da6d7e593f.slice/crio-011dd36353e2799ba79168fa6018f124265668de293200bbc73ff6be63a61d22 WatchSource:0}: Error finding container 011dd36353e2799ba79168fa6018f124265668de293200bbc73ff6be63a61d22: Status 404 returned error can't find the container with id 011dd36353e2799ba79168fa6018f124265668de293200bbc73ff6be63a61d22
Jul 24 17:10:14.725217 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:14.725196    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625c2cb7_a1b2_49c6_800f_91336caf9d82.slice/crio-abde8b7eae025acd6f15036e8ea881b5a718f1769bf9ff43fab1ff36edcebf29 WatchSource:0}: Error finding container abde8b7eae025acd6f15036e8ea881b5a718f1769bf9ff43fab1ff36edcebf29: Status 404 returned error can't find the container with id abde8b7eae025acd6f15036e8ea881b5a718f1769bf9ff43fab1ff36edcebf29
Jul 24 17:10:14.727275 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:14.727155    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5aa1a5c_d2bf_4780_85ca_730789acc3f3.slice/crio-84255c10e619dce983d7b023790c2ebea3405bbbd1f5e0b360b7902eb652cc93 WatchSource:0}: Error finding container 84255c10e619dce983d7b023790c2ebea3405bbbd1f5e0b360b7902eb652cc93: Status 404 returned error can't find the container with id 84255c10e619dce983d7b023790c2ebea3405bbbd1f5e0b360b7902eb652cc93
Jul 24 17:10:14.899155 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:14.898936    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:10:15.116502 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.116457    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:15.116694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.116514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:15.116694 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.116647    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:15.116815 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.116713    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs podName:573221db-ed4c-48f8-8267-b5eeab52090a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:17.116692578 +0000 UTC m=+6.087824739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs") pod "network-metrics-daemon-8qx5c" (UID: "573221db-ed4c-48f8-8267-b5eeab52090a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:15.117163 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.117137    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:15.117300 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.117198    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret podName:8d915ffa-99b0-4ff7-abc6-ca4c0df718bb nodeName:}" failed. No retries permitted until 2026-07-24 17:10:17.11718451 +0000 UTC m=+6.088316675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret") pod "global-pull-secret-syncer-ptxkr" (UID: "8d915ffa-99b0-4ff7-abc6-ca4c0df718bb") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:15.217975 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.217885    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:15.218183 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.218089    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:15.218183 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.218116    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:15.218183 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.218130    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hmhtj for pod openshift-network-diagnostics/network-check-target-th2mm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:15.218359 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.218194    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj podName:d3a36ac0-84b7-4717-be5b-9bca1c9de683 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:17.218173331 +0000 UTC m=+6.189305514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmhtj" (UniqueName: "kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj") pod "network-check-target-th2mm" (UID: "d3a36ac0-84b7-4717-be5b-9bca1c9de683") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:15.431520 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.431202    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:05:14 +0000 UTC" deadline="2028-04-29 18:50:42.530434556 +0000 UTC"
Jul 24 17:10:15.431520 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.431265    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15481h40m27.099173688s"
Jul 24 17:10:15.502272 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.502177    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:15.502437 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.502310    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:15.502747 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.502723    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:15.502840 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.502821    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:15.502904 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.502887    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:15.502982 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:15.502959    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:15.526368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.526328    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" event={"ID":"d240ac75-8247-4228-bcb0-c5f2c09c882f","Type":"ContainerStarted","Data":"cb8cb46ffd7e5ce6810f7e85c02f6a0da6cd38e04b202e97061c1e30626e9ae8"}
Jul 24 17:10:15.534754 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.534716    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal" event={"ID":"9c39321f7468a5658b1cdb40ffa446cd","Type":"ContainerStarted","Data":"ae8cda28a122cd5f34fa7339405e6518828f7cfcfc62f58b9fa8419ef3e5026a"}
Jul 24 17:10:15.545083 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.544924    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"011dd36353e2799ba79168fa6018f124265668de293200bbc73ff6be63a61d22"}
Jul 24 17:10:15.549893 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.549861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-267rb" event={"ID":"a5aa1a5c-d2bf-4780-85ca-730789acc3f3","Type":"ContainerStarted","Data":"84255c10e619dce983d7b023790c2ebea3405bbbd1f5e0b360b7902eb652cc93"}
Jul 24 17:10:15.557045 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.557013    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fq6nj" event={"ID":"625c2cb7-a1b2-49c6-800f-91336caf9d82","Type":"ContainerStarted","Data":"abde8b7eae025acd6f15036e8ea881b5a718f1769bf9ff43fab1ff36edcebf29"}
Jul 24 17:10:15.576096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.576042    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cfbd8" event={"ID":"55f4b489-e97a-4542-914e-f06da6c9e49f","Type":"ContainerStarted","Data":"0670f17f6df626a40f6ce7bc8fa75845e260f58280f160a07e2e2a4599b6857c"}
Jul 24 17:10:15.582562 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.582530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-45lzv" event={"ID":"8ab674fe-1d10-41b0-90ce-eb0ddfae3042","Type":"ContainerStarted","Data":"b9edd3446a9c8c7f457d34ecadff6df34c9d09bcd890a71e799fc87d6e74f594"}
Jul 24 17:10:15.587294 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.587259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bmq8m" event={"ID":"c8135909-83dd-4abb-9121-2c9b7dbd77ec","Type":"ContainerStarted","Data":"78caa5b088fa5bbfe9ce08a1f7ba5781b4ad6592422d67d5a5e2abfb1dfd9cc1"}
Jul 24 17:10:15.596292 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.596261    2562 generic.go:356] "Generic (PLEG): container finished" podID="8d767b07b5a78a429c99353bd6c317f9" containerID="e666a4eed39c9524b3b738629d9f2c45e04c41a95facb9c1994f7c8f558aa798" exitCode=0
Jul 24 17:10:15.596443 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.596362    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal" event={"ID":"8d767b07b5a78a429c99353bd6c317f9","Type":"ContainerDied","Data":"e666a4eed39c9524b3b738629d9f2c45e04c41a95facb9c1994f7c8f558aa798"}
Jul 24 17:10:15.609932 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.609806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4ngr7" event={"ID":"6769c8c9-a903-4966-9eab-351a6ec580fc","Type":"ContainerStarted","Data":"b6274c6393266fc90b3b468e1ed1e42f17fb0de6df2ff27d49e8c948e6504d2b"}
Jul 24 17:10:15.613834 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.613145    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-5.ec2.internal" podStartSLOduration=2.613125321 podStartE2EDuration="2.613125321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:15.553604074 +0000 UTC m=+4.524736260" watchObservedRunningTime="2026-07-24 17:10:15.613125321 +0000 UTC m=+4.584257505"
Jul 24 17:10:15.616846 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:15.616805    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s" event={"ID":"765aa388-ed15-45fd-9f65-7d6204598947","Type":"ContainerStarted","Data":"124b9d3c6fd896e933b005b2b8d5abdc4c61d4825c7d372f7b0b5b6c25bddfb8"}
Jul 24 17:10:16.642559 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:16.642432    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal" event={"ID":"8d767b07b5a78a429c99353bd6c317f9","Type":"ContainerStarted","Data":"a9e246c043d79762768e63c4514124bdadc053d10ac4396b6df30214c2d87816"}
Jul 24 17:10:17.136023 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:17.135162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:17.136023 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:17.135216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:17.136023 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.135372    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:17.136023 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.135439    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs podName:573221db-ed4c-48f8-8267-b5eeab52090a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:21.135420294 +0000 UTC m=+10.106552461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs") pod "network-metrics-daemon-8qx5c" (UID: "573221db-ed4c-48f8-8267-b5eeab52090a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:17.136023 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.135874    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:17.136023 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.135928    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret podName:8d915ffa-99b0-4ff7-abc6-ca4c0df718bb nodeName:}" failed. No retries permitted until 2026-07-24 17:10:21.135911595 +0000 UTC m=+10.107043761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret") pod "global-pull-secret-syncer-ptxkr" (UID: "8d915ffa-99b0-4ff7-abc6-ca4c0df718bb") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:17.237238 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:17.236474    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:17.237238 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.236691    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:17.237238 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.236714    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:17.237238 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.236728    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hmhtj for pod openshift-network-diagnostics/network-check-target-th2mm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:17.237238 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.236803    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj podName:d3a36ac0-84b7-4717-be5b-9bca1c9de683 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:21.2367714 +0000 UTC m=+10.207903580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmhtj" (UniqueName: "kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj") pod "network-check-target-th2mm" (UID: "d3a36ac0-84b7-4717-be5b-9bca1c9de683") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:17.500954 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:17.500185    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:17.500954 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.500325    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:17.500954 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:17.500744    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:17.500954 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.500839    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:17.501565 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:17.501388    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:17.501565 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:17.501518    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:19.500288 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:19.500246    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:19.500784 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:19.500398    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:19.500937 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:19.500850    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:19.501005 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:19.500966    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:19.501152 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:19.501126    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:19.501234 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:19.501219    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:21.176256 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:21.175573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:21.176256 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:21.175626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:21.176256 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.175780    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:21.176256 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.175842    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs podName:573221db-ed4c-48f8-8267-b5eeab52090a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:29.175823217 +0000 UTC m=+18.146955384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs") pod "network-metrics-daemon-8qx5c" (UID: "573221db-ed4c-48f8-8267-b5eeab52090a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:21.176256 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.176218    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:21.176864 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.176314    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret podName:8d915ffa-99b0-4ff7-abc6-ca4c0df718bb nodeName:}" failed. No retries permitted until 2026-07-24 17:10:29.176299207 +0000 UTC m=+18.147431368 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret") pod "global-pull-secret-syncer-ptxkr" (UID: "8d915ffa-99b0-4ff7-abc6-ca4c0df718bb") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:21.276478 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:21.276437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:21.276667 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.276611    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:21.276667 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.276630    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:21.276667 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.276638    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hmhtj for pod openshift-network-diagnostics/network-check-target-th2mm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:21.276773 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.276687    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj podName:d3a36ac0-84b7-4717-be5b-9bca1c9de683 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:29.276673319 +0000 UTC m=+18.247805480 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmhtj" (UniqueName: "kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj") pod "network-check-target-th2mm" (UID: "d3a36ac0-84b7-4717-be5b-9bca1c9de683") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:21.501153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:21.501038    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:21.501330 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.501191    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:21.501493 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:21.501473    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:21.501597 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.501577    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:21.501638 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:21.501597    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:21.501723 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:21.501701    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:23.500239 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:23.500205    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:23.500707 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:23.500355    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:23.500801 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:23.500765    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:23.500856 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:23.500830    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:23.500896 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:23.500890    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:23.501071 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:23.500967    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:25.499679 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:25.499633    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:25.500089 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:25.499782    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:25.500212 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:25.500192    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:25.500325 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:25.500297    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:25.500437 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:25.500357    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:25.500437 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:25.500419    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:27.499360 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:27.499323    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:27.499360 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:27.499365    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:27.499860 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:27.499323    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:27.499860 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:27.499462    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:27.499860 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:27.499580    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:27.499860 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:27.499669    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:29.239485 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:29.239437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:29.239944 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:29.239495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:29.239944 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.239644    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:29.239944 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.239708    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs podName:573221db-ed4c-48f8-8267-b5eeab52090a nodeName:}" failed. No retries permitted until 2026-07-24 17:10:45.239688949 +0000 UTC m=+34.210821118 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs") pod "network-metrics-daemon-8qx5c" (UID: "573221db-ed4c-48f8-8267-b5eeab52090a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:29.240167 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.240149    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:29.240266 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.240204    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret podName:8d915ffa-99b0-4ff7-abc6-ca4c0df718bb nodeName:}" failed. No retries permitted until 2026-07-24 17:10:45.240188725 +0000 UTC m=+34.211320902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret") pod "global-pull-secret-syncer-ptxkr" (UID: "8d915ffa-99b0-4ff7-abc6-ca4c0df718bb") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:29.340296 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:29.340264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:29.340450 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.340384    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:29.340450 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.340400    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:29.340450 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.340409    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hmhtj for pod openshift-network-diagnostics/network-check-target-th2mm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:29.340570 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.340459    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj podName:d3a36ac0-84b7-4717-be5b-9bca1c9de683 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:45.340444133 +0000 UTC m=+34.311576291 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmhtj" (UniqueName: "kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj") pod "network-check-target-th2mm" (UID: "d3a36ac0-84b7-4717-be5b-9bca1c9de683") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:29.500090 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:29.500007    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:29.500090 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:29.500010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:29.500259 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.500130    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:29.500259 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:29.500006    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:29.500259 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.500221    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:29.500428 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:29.500292    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:31.504081 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:31.504034    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:31.504498 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:31.504182    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:31.504498 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:31.504235    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:31.504498 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:31.504303    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:31.504633 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:31.504587    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:31.504858 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:31.504825    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:33.500065 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:33.500006    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:33.500576 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:33.500006    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:33.500576 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:33.500129    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:33.500576 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:33.500032    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:33.500576 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:33.500227    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:33.500576 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:33.500325    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:34.678421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.677947    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-45lzv" event={"ID":"8ab674fe-1d10-41b0-90ce-eb0ddfae3042","Type":"ContainerStarted","Data":"4398976b93f7ef724ea5c318dbeceaeab9ba1d31d4fa3ac97a4eb8ab76a9b398"}
Jul 24 17:10:34.679401 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.679376    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bmq8m" event={"ID":"c8135909-83dd-4abb-9121-2c9b7dbd77ec","Type":"ContainerStarted","Data":"87b7233877297a3ec3f0a8f833ca29f6e6382f5df5d20da8dde86195bddff38e"}
Jul 24 17:10:34.680690 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.680668    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4ngr7" event={"ID":"6769c8c9-a903-4966-9eab-351a6ec580fc","Type":"ContainerStarted","Data":"6faa6eee25c017f267b2189c75c0b688cea761a3155b382e40512684c955c5af"}
Jul 24 17:10:34.682306 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.682273    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s" event={"ID":"765aa388-ed15-45fd-9f65-7d6204598947","Type":"ContainerStarted","Data":"253a69f0ad36cb5171945ff117ba277f822c36d34895dc1b0b430e9b91885310"}
Jul 24 17:10:34.683738 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.683714    2562 generic.go:356] "Generic (PLEG): container finished" podID="d240ac75-8247-4228-bcb0-c5f2c09c882f" containerID="64f6110abd3b4f157e1ba355ec9dbb77b8528c760374b6da6cc60bb7d0166966" exitCode=0
Jul 24 17:10:34.683830 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.683779    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" event={"ID":"d240ac75-8247-4228-bcb0-c5f2c09c882f","Type":"ContainerDied","Data":"64f6110abd3b4f157e1ba355ec9dbb77b8528c760374b6da6cc60bb7d0166966"}
Jul 24 17:10:34.686658 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.686641    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:10:34.686987 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.686965    2562 generic.go:356] "Generic (PLEG): container finished" podID="697f9852-7423-45f4-851e-89da6d7e593f" containerID="83d4adbe7340a634c17b55b6566e5b2e8c7d49d94a4b12f5fcc5d1b9ca23408d" exitCode=1
Jul 24 17:10:34.687095 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.687053    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"e7d82e69f3567292155bc4c934ffedd88feb1fb420477a3320aa3e3f2baa9bba"}
Jul 24 17:10:34.687153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.687129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"80f2a859d37133f6cb56e5296bbce49bdc058d7c50c78feed591d7dbc886c7e4"}
Jul 24 17:10:34.687235 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.687180    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"7a4c9626b51604e06982d6429aa2f592d42ca3c478553034db3b5f9ed5d74a1e"}
Jul 24 17:10:34.687235 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.687223    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"5fcd7448b00127de77b395721cd66b47a8fb273e4bd5dd3783772597a2d33088"}
Jul 24 17:10:34.687307 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.687268    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerDied","Data":"83d4adbe7340a634c17b55b6566e5b2e8c7d49d94a4b12f5fcc5d1b9ca23408d"}
Jul 24 17:10:34.687445 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.687320    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"f00fe5ee621574c667189cadb73ae197e7b4f57021308c3747b141ab1c98d7db"}
Jul 24 17:10:34.688603 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.688582    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fq6nj" event={"ID":"625c2cb7-a1b2-49c6-800f-91336caf9d82","Type":"ContainerStarted","Data":"1550b3cf8992db3555f28323ef5c298fdf2dba81164c3d8dc147fd83b1c38dd8"}
Jul 24 17:10:34.689952 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.689934    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cfbd8" event={"ID":"55f4b489-e97a-4542-914e-f06da6c9e49f","Type":"ContainerStarted","Data":"d8138fa892e0ac4e10863efaaeae709cd4f3387188e589f886a28398f0f2b9f8"}
Jul 24 17:10:34.694204 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.694171    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-5.ec2.internal" podStartSLOduration=21.694157567 podStartE2EDuration="21.694157567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:16.659785299 +0000 UTC m=+5.630917488" watchObservedRunningTime="2026-07-24 17:10:34.694157567 +0000 UTC m=+23.665289749"
Jul 24 17:10:34.713982 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.713938    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-45lzv" podStartSLOduration=3.758378976 podStartE2EDuration="22.71392327s" totalImagesPullingTime="18.955544294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:34.694471815 +0000 UTC m=+23.665604008" watchObservedRunningTime="2026-07-24 17:10:34.71392327 +0000 UTC m=+23.685055451"
Jul 24 17:10:34.727950 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.727908    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bmq8m" podStartSLOduration=3.770233204 podStartE2EDuration="22.727894084s" totalImagesPullingTime="18.95766088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:34.727795048 +0000 UTC m=+23.698927228" watchObservedRunningTime="2026-07-24 17:10:34.727894084 +0000 UTC m=+23.699026264"
Jul 24 17:10:34.746275 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.746230    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cfbd8" podStartSLOduration=3.554965956 podStartE2EDuration="22.746218462s" totalImagesPullingTime="19.191252506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:34.745896305 +0000 UTC m=+23.717028487" watchObservedRunningTime="2026-07-24 17:10:34.746218462 +0000 UTC m=+23.717350642"
Jul 24 17:10:34.761686 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.761636    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fq6nj" podStartSLOduration=3.831495776 podStartE2EDuration="22.761620483s" totalImagesPullingTime="18.930124707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:34.761617449 +0000 UTC m=+23.732749631" watchObservedRunningTime="2026-07-24 17:10:34.761620483 +0000 UTC m=+23.732752665"
Jul 24 17:10:34.776342 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:34.776302    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4ngr7" podStartSLOduration=8.155253616 podStartE2EDuration="22.776288541s" totalImagesPullingTime="14.621034925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:34.776107905 +0000 UTC m=+23.747240085" watchObservedRunningTime="2026-07-24 17:10:34.776288541 +0000 UTC m=+23.747420743"
Jul 24 17:10:35.190275 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.190245    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 17:10:35.477480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.477354    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T17:10:35.190270711Z","UUID":"07d5a119-b2b8-4606-98f7-021b1aefa116","Handler":null,"Name":"","Endpoint":""}
Jul 24 17:10:35.479113 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.479092    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 17:10:35.479236 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.479137    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 17:10:35.499963 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.499876    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:35.499963 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.499889    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:35.500165 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.500010    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:35.500165 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:35.500029    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:35.500165 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:35.500124    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:35.500294 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:35.500185    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:35.694169 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.694084    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s" event={"ID":"765aa388-ed15-45fd-9f65-7d6204598947","Type":"ContainerStarted","Data":"11f52b7be328cda4ebe107cfbf6856e33794dad551a83ceee548115034ab47e4"}
Jul 24 17:10:35.695610 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.695579    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-267rb" event={"ID":"a5aa1a5c-d2bf-4780-85ca-730789acc3f3","Type":"ContainerStarted","Data":"70abd8966f3354ee3cc689d9487055109b6f6f763cbb59ad7f0ea91d47fa4df4"}
Jul 24 17:10:35.713163 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.713104    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-267rb" podStartSLOduration=4.80213771 podStartE2EDuration="23.713082524s" totalImagesPullingTime="18.910944814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:35.711863749 +0000 UTC m=+24.682995931" watchObservedRunningTime="2026-07-24 17:10:35.713082524 +0000 UTC m=+24.684214707"
Jul 24 17:10:35.803595 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.803499    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:35.804163 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:35.804140    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:36.702153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:36.702122    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:10:36.702658 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:36.702631    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"556cb42423a4d7ea4051532560f5e08f90df9eff6b844b6aecd82fd63b5254f4"}
Jul 24 17:10:36.704967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:36.704897    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s" event={"ID":"765aa388-ed15-45fd-9f65-7d6204598947","Type":"ContainerStarted","Data":"386f0579b649493b23340ccef76f70a47662b51a45759ee9fff8f13bf729e219"}
Jul 24 17:10:36.705446 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:36.705277    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:36.705786 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:36.705772    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4ngr7"
Jul 24 17:10:36.723356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:36.723304    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v722s" podStartSLOduration=3.641472951 podStartE2EDuration="24.723291714s" totalImagesPullingTime="21.081818763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:36.72300026 +0000 UTC m=+25.694132440" watchObservedRunningTime="2026-07-24 17:10:36.723291714 +0000 UTC m=+25.694423894"
Jul 24 17:10:37.501329 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:37.501069    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:37.501504 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:37.501111    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:37.501504 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:37.501443    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:37.501504 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:37.501125    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:37.501620 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:37.501522    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:37.501620 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:37.501570    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:39.504325 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.503972    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:39.504983 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.503972    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:39.504983 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.503981    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:39.504983 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:39.504505    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:39.504983 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:39.504469    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:39.504983 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:39.504591    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:39.714254 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.714217    2562 generic.go:356] "Generic (PLEG): container finished" podID="d240ac75-8247-4228-bcb0-c5f2c09c882f" containerID="e0974f128a954cd8b46c2c843488501e4d738a4aec80c5ef69c9a32499f4b562" exitCode=0
Jul 24 17:10:39.714408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.714301    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" event={"ID":"d240ac75-8247-4228-bcb0-c5f2c09c882f","Type":"ContainerDied","Data":"e0974f128a954cd8b46c2c843488501e4d738a4aec80c5ef69c9a32499f4b562"}
Jul 24 17:10:39.717694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.717679    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:10:39.718101 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.718078    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"509e716705c53019554191386c1fd84fdd908052f63b107d0947e8a8e672b8f1"}
Jul 24 17:10:39.718347 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.718327    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:39.718431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.718404    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:39.718473 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.718432    2562 scope.go:117] "RemoveContainer" containerID="83d4adbe7340a634c17b55b6566e5b2e8c7d49d94a4b12f5fcc5d1b9ca23408d"
Jul 24 17:10:39.734444 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:39.734427    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:40.722707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:40.722615    2562 generic.go:356] "Generic (PLEG): container finished" podID="d240ac75-8247-4228-bcb0-c5f2c09c882f" containerID="8ca11786717873351cf5f7d741e33ba6f25837462f2234810365ebdc156eeb01" exitCode=0
Jul 24 17:10:40.723186 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:40.722711    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" event={"ID":"d240ac75-8247-4228-bcb0-c5f2c09c882f","Type":"ContainerDied","Data":"8ca11786717873351cf5f7d741e33ba6f25837462f2234810365ebdc156eeb01"}
Jul 24 17:10:40.726653 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:40.726632    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:10:40.727044 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:40.726997    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" event={"ID":"697f9852-7423-45f4-851e-89da6d7e593f","Type":"ContainerStarted","Data":"e4d2e6e35050a8cd4b3b14ed4b42c41af086a9d7ad9ccb76b2ea66579978ae77"}
Jul 24 17:10:40.727208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:40.727194    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:40.742667 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:40.742639    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:10:40.768205 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:40.768162    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk" podStartSLOduration=9.779117106 podStartE2EDuration="28.768146562s" totalImagesPullingTime="18.989029456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:40.766573442 +0000 UTC m=+29.737705635" watchObservedRunningTime="2026-07-24 17:10:40.768146562 +0000 UTC m=+29.739278744"
Jul 24 17:10:41.432458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:41.432278    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8qx5c"]
Jul 24 17:10:41.432597 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:41.432584    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:41.432731 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:41.432695    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:41.435209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:41.435187    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ptxkr"]
Jul 24 17:10:41.435380 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:41.435284    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:41.435429 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:41.435377    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:41.435999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:41.435981    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-th2mm"]
Jul 24 17:10:41.436118 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:41.436105    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:41.436228 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:41.436207    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:41.731255 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:41.731213    2562 generic.go:356] "Generic (PLEG): container finished" podID="d240ac75-8247-4228-bcb0-c5f2c09c882f" containerID="1ec307b26d6a136e2bb83b5ea8345a39e79eac071e04bf744de942afef98b0a7" exitCode=0
Jul 24 17:10:41.731585 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:41.731301    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" event={"ID":"d240ac75-8247-4228-bcb0-c5f2c09c882f","Type":"ContainerDied","Data":"1ec307b26d6a136e2bb83b5ea8345a39e79eac071e04bf744de942afef98b0a7"}
Jul 24 17:10:43.414095 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:43.414007    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:43.499634 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:43.499604    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:43.499794 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:43.499659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:43.499794 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:43.499669    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:43.499794 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:43.499761    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:43.500072 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:43.500005    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:43.500209 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:43.500166    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:43.520548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:43.520517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:44.915585 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:44.915537    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:44.963710 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:44.963678    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:45.252365 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:45.252333    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:45.252580 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:45.252379    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:45.252580 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.252525    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:45.252580 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.252553    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:45.252723 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.252601    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret podName:8d915ffa-99b0-4ff7-abc6-ca4c0df718bb nodeName:}" failed. No retries permitted until 2026-07-24 17:11:17.252579635 +0000 UTC m=+66.223711799 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret") pod "global-pull-secret-syncer-ptxkr" (UID: "8d915ffa-99b0-4ff7-abc6-ca4c0df718bb") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:10:45.252723 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.252624    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs podName:573221db-ed4c-48f8-8267-b5eeab52090a nodeName:}" failed. No retries permitted until 2026-07-24 17:11:17.252612913 +0000 UTC m=+66.223745079 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs") pod "network-metrics-daemon-8qx5c" (UID: "573221db-ed4c-48f8-8267-b5eeab52090a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:10:45.353035 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:45.352991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:45.353205 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.353185    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:10:45.353251 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.353208    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:10:45.353251 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.353221    2562 projected.go:196] Error preparing data for projected volume kube-api-access-hmhtj for pod openshift-network-diagnostics/network-check-target-th2mm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:45.353316 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.353286    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj podName:d3a36ac0-84b7-4717-be5b-9bca1c9de683 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:17.353265902 +0000 UTC m=+66.324398065 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hmhtj" (UniqueName: "kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj") pod "network-check-target-th2mm" (UID: "d3a36ac0-84b7-4717-be5b-9bca1c9de683") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:10:45.500243 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:45.500208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:45.500418 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:45.500208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:45.500418 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.500345    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8qx5c" podUID="573221db-ed4c-48f8-8267-b5eeab52090a"
Jul 24 17:10:45.500512 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.500411    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-th2mm" podUID="d3a36ac0-84b7-4717-be5b-9bca1c9de683"
Jul 24 17:10:45.500512 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:45.500212    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:45.500588 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:45.500520    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ptxkr" podUID="8d915ffa-99b0-4ff7-abc6-ca4c0df718bb"
Jul 24 17:10:46.745740 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.745708    2562 generic.go:356] "Generic (PLEG): container finished" podID="d240ac75-8247-4228-bcb0-c5f2c09c882f" containerID="75b8db439e0fcaeef5aebf3cedc0f49a7ef3d5afdf68ef3161df2aba828a13a0" exitCode=0
Jul 24 17:10:46.746269 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.745747    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" event={"ID":"d240ac75-8247-4228-bcb0-c5f2c09c882f","Type":"ContainerDied","Data":"75b8db439e0fcaeef5aebf3cedc0f49a7ef3d5afdf68ef3161df2aba828a13a0"}
Jul 24 17:10:46.870494 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.870467    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-5.ec2.internal" event="NodeReady"
Jul 24 17:10:46.870651 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.870625    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 17:10:46.904374 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.904349    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d474986c-7zdg2"]
Jul 24 17:10:46.925686 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.925664    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v8khf"]
Jul 24 17:10:46.925842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.925811    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:46.929525 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.929504    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 17:10:46.929623 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.929507    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 17:10:46.929623 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.929594    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 17:10:46.929707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.929688    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g65mg\""
Jul 24 17:10:46.938261 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.938240    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 17:10:46.943951 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.943929    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hts7p"]
Jul 24 17:10:46.944077 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.944056    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:46.946670 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.946650    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 17:10:46.946742 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.946708    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 17:10:46.947147 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.947128    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cpjrs\""
Jul 24 17:10:46.958378 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.958355    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hts7p"
Jul 24 17:10:46.958471 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.958379    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d474986c-7zdg2"]
Jul 24 17:10:46.958471 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.958440    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v8khf"]
Jul 24 17:10:46.958471 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.958459    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hts7p"]
Jul 24 17:10:46.960949 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.960933    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 17:10:46.961054 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.960952    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 17:10:46.961129 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.961112    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 17:10:46.961206 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:46.961183    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g5d6l\""
Jul 24 17:10:47.039637 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.039578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:47.066440 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lmlv\" (UniqueName: \"kubernetes.io/projected/dde92a18-877a-423d-9e1b-ed03e6638e03-kube-api-access-7lmlv\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.066510 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066469    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dde92a18-877a-423d-9e1b-ed03e6638e03-config-volume\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.066510 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.066575 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-image-registry-private-configuration\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.066610 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572b70ea-b429-4a02-995a-81eb41af79c8-ca-trust-extracted\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.066610 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066596    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-registry-certificates\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.066674 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066612    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-trusted-ca\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.066674 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066627    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-installation-pull-secrets\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.066674 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnpz6\" (UniqueName: \"kubernetes.io/projected/11a91422-52c6-44f1-b193-7f2ed40394ca-kube-api-access-jnpz6\") pod \"ingress-canary-hts7p\" (UID: \"11a91422-52c6-44f1-b193-7f2ed40394ca\") " pod="openshift-ingress-canary/ingress-canary-hts7p"
Jul 24 17:10:47.066759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11a91422-52c6-44f1-b193-7f2ed40394ca-cert\") pod \"ingress-canary-hts7p\" (UID: \"11a91422-52c6-44f1-b193-7f2ed40394ca\") " pod="openshift-ingress-canary/ingress-canary-hts7p"
Jul 24 17:10:47.066759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066722    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.066759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066749    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk6n5\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-kube-api-access-qk6n5\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.066850 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066784    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dde92a18-877a-423d-9e1b-ed03e6638e03-tmp-dir\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.066850 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.066809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-bound-sa-token\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.167761 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167721    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dde92a18-877a-423d-9e1b-ed03e6638e03-config-volume\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.167761 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.167990 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167781    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-image-registry-private-configuration\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.167990 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572b70ea-b429-4a02-995a-81eb41af79c8-ca-trust-extracted\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.167990 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167862    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-registry-certificates\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.167990 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167885    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-trusted-ca\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.167990 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.167888    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:10:47.167990 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-installation-pull-secrets\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.167990 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167941    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnpz6\" (UniqueName: \"kubernetes.io/projected/11a91422-52c6-44f1-b193-7f2ed40394ca-kube-api-access-jnpz6\") pod \"ingress-canary-hts7p\" (UID: \"11a91422-52c6-44f1-b193-7f2ed40394ca\") " pod="openshift-ingress-canary/ingress-canary-hts7p"
Jul 24 17:10:47.167990 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.167965    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls podName:dde92a18-877a-423d-9e1b-ed03e6638e03 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:47.667944963 +0000 UTC m=+36.639077122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls") pod "dns-default-v8khf" (UID: "dde92a18-877a-423d-9e1b-ed03e6638e03") : secret "dns-default-metrics-tls" not found
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.167995    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11a91422-52c6-44f1-b193-7f2ed40394ca-cert\") pod \"ingress-canary-hts7p\" (UID: \"11a91422-52c6-44f1-b193-7f2ed40394ca\") " pod="openshift-ingress-canary/ingress-canary-hts7p"
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk6n5\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-kube-api-access-qk6n5\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dde92a18-877a-423d-9e1b-ed03e6638e03-tmp-dir\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168114    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-bound-sa-token\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lmlv\" (UniqueName: \"kubernetes.io/projected/dde92a18-877a-423d-9e1b-ed03e6638e03-kube-api-access-7lmlv\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572b70ea-b429-4a02-995a-81eb41af79c8-ca-trust-extracted\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.168371    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.168385    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d474986c-7zdg2: secret "image-registry-tls" not found
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.168448    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls podName:572b70ea-b429-4a02-995a-81eb41af79c8 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:47.668430371 +0000 UTC m=+36.639562552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls") pod "image-registry-5d474986c-7zdg2" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8") : secret "image-registry-tls" not found
Jul 24 17:10:47.168488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168472    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dde92a18-877a-423d-9e1b-ed03e6638e03-config-volume\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.168906 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168616    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-registry-certificates\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.168906 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.168724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dde92a18-877a-423d-9e1b-ed03e6638e03-tmp-dir\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.169367 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.169346    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-trusted-ca\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.172436 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.172417    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-installation-pull-secrets\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.172491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.172447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/11a91422-52c6-44f1-b193-7f2ed40394ca-cert\") pod \"ingress-canary-hts7p\" (UID: \"11a91422-52c6-44f1-b193-7f2ed40394ca\") " pod="openshift-ingress-canary/ingress-canary-hts7p"
Jul 24 17:10:47.172491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.172462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-image-registry-private-configuration\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.176839 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.176812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-bound-sa-token\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.177145 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.177129    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk6n5\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-kube-api-access-qk6n5\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.177574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.177554    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lmlv\" (UniqueName: \"kubernetes.io/projected/dde92a18-877a-423d-9e1b-ed03e6638e03-kube-api-access-7lmlv\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.178172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.178154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnpz6\" (UniqueName: \"kubernetes.io/projected/11a91422-52c6-44f1-b193-7f2ed40394ca-kube-api-access-jnpz6\") pod \"ingress-canary-hts7p\" (UID: \"11a91422-52c6-44f1-b193-7f2ed40394ca\") " pod="openshift-ingress-canary/ingress-canary-hts7p"
Jul 24 17:10:47.296797 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.296717    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hts7p"
Jul 24 17:10:47.476965 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.476795    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hts7p"]
Jul 24 17:10:47.480294 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:47.480268    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a91422_52c6_44f1_b193_7f2ed40394ca.slice/crio-28243b3773689176d56fd6ef126548506b3650bb504238e3c1b02cd980016cd0 WatchSource:0}: Error finding container 28243b3773689176d56fd6ef126548506b3650bb504238e3c1b02cd980016cd0: Status 404 returned error can't find the container with id 28243b3773689176d56fd6ef126548506b3650bb504238e3c1b02cd980016cd0
Jul 24 17:10:47.499444 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.499422    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:10:47.499444 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.499434    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:10:47.499594 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.499440    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:10:47.502264 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.502244    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:10:47.502264 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.502255    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:10:47.502419 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.502266    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-56frs\""
Jul 24 17:10:47.502419 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.502293    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gkgpl\""
Jul 24 17:10:47.502419 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.502292    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 17:10:47.502419 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.502368    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 17:10:47.673655 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.673554    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:47.673655 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.673612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:47.673871 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.673692    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:10:47.673871 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.673700    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:10:47.673871 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.673713    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d474986c-7zdg2: secret "image-registry-tls" not found
Jul 24 17:10:47.673871 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.673751    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls podName:dde92a18-877a-423d-9e1b-ed03e6638e03 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:48.673736255 +0000 UTC m=+37.644868418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls") pod "dns-default-v8khf" (UID: "dde92a18-877a-423d-9e1b-ed03e6638e03") : secret "dns-default-metrics-tls" not found
Jul 24 17:10:47.673871 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:47.673764    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls podName:572b70ea-b429-4a02-995a-81eb41af79c8 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:48.673758764 +0000 UTC m=+37.644890926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls") pod "image-registry-5d474986c-7zdg2" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8") : secret "image-registry-tls" not found
Jul 24 17:10:47.751431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.751400    2562 generic.go:356] "Generic (PLEG): container finished" podID="d240ac75-8247-4228-bcb0-c5f2c09c882f" containerID="20914c386e60ae7251c39d403e8af512cb3f875c6c2bcd25d15c8f22c5256853" exitCode=0
Jul 24 17:10:47.751832 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.751485    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" event={"ID":"d240ac75-8247-4228-bcb0-c5f2c09c882f","Type":"ContainerDied","Data":"20914c386e60ae7251c39d403e8af512cb3f875c6c2bcd25d15c8f22c5256853"}
Jul 24 17:10:47.752666 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:47.752639    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hts7p" event={"ID":"11a91422-52c6-44f1-b193-7f2ed40394ca","Type":"ContainerStarted","Data":"28243b3773689176d56fd6ef126548506b3650bb504238e3c1b02cd980016cd0"}
Jul 24 17:10:48.442353 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:48.442315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:48.510460 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:48.510420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:48.680828 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:48.680794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:48.681009 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:48.680854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:48.681009 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:48.680966    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:10:48.681009 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:48.680991    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:10:48.681009 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:48.681005    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d474986c-7zdg2: secret "image-registry-tls" not found
Jul 24 17:10:48.681231 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:48.681083    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls podName:dde92a18-877a-423d-9e1b-ed03e6638e03 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:50.681062205 +0000 UTC m=+39.652194379 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls") pod "dns-default-v8khf" (UID: "dde92a18-877a-423d-9e1b-ed03e6638e03") : secret "dns-default-metrics-tls" not found
Jul 24 17:10:48.681231 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:48.681105    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls podName:572b70ea-b429-4a02-995a-81eb41af79c8 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:50.681093418 +0000 UTC m=+39.652225583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls") pod "image-registry-5d474986c-7zdg2" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8") : secret "image-registry-tls" not found
Jul 24 17:10:48.761523 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:48.761476    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" event={"ID":"d240ac75-8247-4228-bcb0-c5f2c09c882f","Type":"ContainerStarted","Data":"f12d3651407178c06a828232fff8c803de91d83340115c5646d34d48865187d1"}
Jul 24 17:10:48.786433 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:48.786310    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2lfq6" podStartSLOduration=7.509306184 podStartE2EDuration="36.786290894s" totalImagesPullingTime="29.27698471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:48.784796428 +0000 UTC m=+37.755928606" watchObservedRunningTime="2026-07-24 17:10:48.786290894 +0000 UTC m=+37.757423076"
Jul 24 17:10:48.906043 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:48.905990    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:48.961883 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:48.961846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:50.697126 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:50.696843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:50.697739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:50.697223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:50.697739 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:50.697388    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:10:50.697739 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:50.697445    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls podName:dde92a18-877a-423d-9e1b-ed03e6638e03 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:54.697428188 +0000 UTC m=+43.668560348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls") pod "dns-default-v8khf" (UID: "dde92a18-877a-423d-9e1b-ed03e6638e03") : secret "dns-default-metrics-tls" not found
Jul 24 17:10:50.697739 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:50.697614    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:10:50.697739 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:50.697637    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d474986c-7zdg2: secret "image-registry-tls" not found
Jul 24 17:10:50.697739 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:50.697690    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls podName:572b70ea-b429-4a02-995a-81eb41af79c8 nodeName:}" failed. No retries permitted until 2026-07-24 17:10:54.697675874 +0000 UTC m=+43.668808032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls") pod "image-registry-5d474986c-7zdg2" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8") : secret "image-registry-tls" not found
Jul 24 17:10:50.767233 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:50.767196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hts7p" event={"ID":"11a91422-52c6-44f1-b193-7f2ed40394ca","Type":"ContainerStarted","Data":"2f27f9a02ea73cb53ec46d3e61b579122f487b5938759414ef595a5bbfdc0e71"}
Jul 24 17:10:52.250179 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:52.250132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:54.176854 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.176799    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hts7p" podStartSLOduration=5.79437572 podStartE2EDuration="8.176782415s" totalImagesPullingTime="2.382406695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:50.783608246 +0000 UTC m=+39.754740427" watchObservedRunningTime="2026-07-24 17:10:54.176782415 +0000 UTC m=+43.147914646"
Jul 24 17:10:54.177627 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.177606    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6c59z"]
Jul 24 17:10:54.199574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.199544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:54.210771 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.210752    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6c59z"]
Jul 24 17:10:54.210873 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.210861    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.213314 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.213272    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 17:10:54.213432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.213416    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bwrlh\""
Jul 24 17:10:54.214499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.214477    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 17:10:54.214499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.214487    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 17:10:54.214499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.214499    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 17:10:54.255503 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.255455    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:54.323128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.323102    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-data-volume\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.323267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.323133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-crio-socket\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.323267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.323222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.323350 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.323333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.323385 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.323359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xrww\" (UniqueName: \"kubernetes.io/projected/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-api-access-8xrww\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.423927 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.423894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-crio-socket\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.424099 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.423953    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.424099 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.424039    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.424099 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.424064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xrww\" (UniqueName: \"kubernetes.io/projected/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-api-access-8xrww\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.424259 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.424103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-data-volume\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.424259 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.424115    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:10:54.424259 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.424156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-crio-socket\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.424259 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.424186    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:10:54.424259 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.424191    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:10:54.924174921 +0000 UTC m=+43.895307080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:10:54.424430 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.424268    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:10:54.924248784 +0000 UTC m=+43.895380961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : configmap "kube-rbac-proxy" not found
Jul 24 17:10:54.424430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.424392    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-data-volume\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.435585 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.435534    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xrww\" (UniqueName: \"kubernetes.io/projected/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-api-access-8xrww\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.625435 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.625393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:54.682458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.682431    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:54.725832 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.725803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:10:54.725965 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.725904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:10:54.725965 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.725956    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:10:54.726062 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.725972    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d474986c-7zdg2: secret "image-registry-tls" not found
Jul 24 17:10:54.726062 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.726007    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:10:54.726062 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.726049    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls podName:572b70ea-b429-4a02-995a-81eb41af79c8 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:02.726033993 +0000 UTC m=+51.697166153 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls") pod "image-registry-5d474986c-7zdg2" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8") : secret "image-registry-tls" not found
Jul 24 17:10:54.726158 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.726079    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls podName:dde92a18-877a-423d-9e1b-ed03e6638e03 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:02.726066243 +0000 UTC m=+51.697198402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls") pod "dns-default-v8khf" (UID: "dde92a18-877a-423d-9e1b-ed03e6638e03") : secret "dns-default-metrics-tls" not found
Jul 24 17:10:54.927142 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.927111    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.927301 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.927177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:54.927301 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.927251    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:10:54.927301 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.927266    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:10:54.927393 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.927317    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:10:55.927302426 +0000 UTC m=+44.898434589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : configmap "kube-rbac-proxy" not found
Jul 24 17:10:54.927393 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:54.927330    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:10:55.927324668 +0000 UTC m=+44.898456827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:10:54.974711 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.974679    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7"]
Jul 24 17:10:54.978348 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.978274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:54.990195 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.990173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7"]
Jul 24 17:10:54.990312 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.990264    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7"
Jul 24 17:10:54.992811 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.992790    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 24 17:10:54.992917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.992828    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 24 17:10:54.993006 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:54.992792    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tnk84\""
Jul 24 17:10:55.041096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.041060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:55.129312 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.129273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2cvx\" (UniqueName: \"kubernetes.io/projected/7a2e3f2b-769a-4187-ba23-ce268bdb7d9b-kube-api-access-w2cvx\") pod \"migrator-6865c9967-4qbc7\" (UID: \"7a2e3f2b-769a-4187-ba23-ce268bdb7d9b\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7"
Jul 24 17:10:55.141690 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.141650    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:55.230372 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.230282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2cvx\" (UniqueName: \"kubernetes.io/projected/7a2e3f2b-769a-4187-ba23-ce268bdb7d9b-kube-api-access-w2cvx\") pod \"migrator-6865c9967-4qbc7\" (UID: \"7a2e3f2b-769a-4187-ba23-ce268bdb7d9b\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7"
Jul 24 17:10:55.239456 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.239434    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2cvx\" (UniqueName: \"kubernetes.io/projected/7a2e3f2b-769a-4187-ba23-ce268bdb7d9b-kube-api-access-w2cvx\") pod \"migrator-6865c9967-4qbc7\" (UID: \"7a2e3f2b-769a-4187-ba23-ce268bdb7d9b\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7"
Jul 24 17:10:55.299468 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.299441    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7"
Jul 24 17:10:55.336637 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.336599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:55.436208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.436177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log" limit=1024
Jul 24 17:10:55.452568 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.452541    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7"]
Jul 24 17:10:55.455889 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:55.455857    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2e3f2b_769a_4187_ba23_ce268bdb7d9b.slice/crio-b93b7c7b6684e24edba308827914c95105e618bf577ab82022f122223fee9a44 WatchSource:0}: Error finding container b93b7c7b6684e24edba308827914c95105e618bf577ab82022f122223fee9a44: Status 404 returned error can't find the container with id b93b7c7b6684e24edba308827914c95105e618bf577ab82022f122223fee9a44
Jul 24 17:10:55.557777 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.557707    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bmq8m_c8135909-83dd-4abb-9121-2c9b7dbd77ec/dns-node-resolver/0.log"
Jul 24 17:10:55.779894 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.779860    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7" event={"ID":"7a2e3f2b-769a-4187-ba23-ce268bdb7d9b","Type":"ContainerStarted","Data":"b93b7c7b6684e24edba308827914c95105e618bf577ab82022f122223fee9a44"}
Jul 24 17:10:55.934888 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.934818    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:55.934888 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:55.934881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:55.935077 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:55.934941    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:10:55.935077 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:55.934976    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:10:55.935077 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:55.935000    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:10:57.934986034 +0000 UTC m=+46.906118196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : configmap "kube-rbac-proxy" not found
Jul 24 17:10:55.935077 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:55.935035    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:10:57.935006672 +0000 UTC m=+46.906138830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:10:56.558197 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.558163    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-45lzv_8ab674fe-1d10-41b0-90ce-eb0ddfae3042/node-ca/0.log"
Jul 24 17:10:56.932277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.932213    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-f26cr"]
Jul 24 17:10:56.955529 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.955506    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-f26cr"]
Jul 24 17:10:56.955656 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.955597    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:56.958210 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.958189    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 24 17:10:56.958325 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.958210    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 24 17:10:56.958325 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.958244    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gz9dx\""
Jul 24 17:10:56.959320 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.959304    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 24 17:10:56.959405 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:56.959367    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 24 17:10:57.046872 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.046848    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91eccd70-9a48-48d6-82cf-b634314cb67a-signing-key\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.046981 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.046896    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91eccd70-9a48-48d6-82cf-b634314cb67a-signing-cabundle\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.046981 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.046919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgg9b\" (UniqueName: \"kubernetes.io/projected/91eccd70-9a48-48d6-82cf-b634314cb67a-kube-api-access-rgg9b\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.147958 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.147930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91eccd70-9a48-48d6-82cf-b634314cb67a-signing-key\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.148076 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.147974    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91eccd70-9a48-48d6-82cf-b634314cb67a-signing-cabundle\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.148076 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.147996    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgg9b\" (UniqueName: \"kubernetes.io/projected/91eccd70-9a48-48d6-82cf-b634314cb67a-kube-api-access-rgg9b\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.148677 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.148660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/91eccd70-9a48-48d6-82cf-b634314cb67a-signing-cabundle\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.150343 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.150326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/91eccd70-9a48-48d6-82cf-b634314cb67a-signing-key\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.156556 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.156527    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgg9b\" (UniqueName: \"kubernetes.io/projected/91eccd70-9a48-48d6-82cf-b634314cb67a-kube-api-access-rgg9b\") pod \"service-ca-658d7b7c78-f26cr\" (UID: \"91eccd70-9a48-48d6-82cf-b634314cb67a\") " pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.264140 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.264102    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-f26cr"
Jul 24 17:10:57.406733 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.406708    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-f26cr"]
Jul 24 17:10:57.407011 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:10:57.406992    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91eccd70_9a48_48d6_82cf_b634314cb67a.slice/crio-9a5021ca876b714cb8a21c4aea775ab19a8bb699b5b2395fd62feac7f6f3f848 WatchSource:0}: Error finding container 9a5021ca876b714cb8a21c4aea775ab19a8bb699b5b2395fd62feac7f6f3f848: Status 404 returned error can't find the container with id 9a5021ca876b714cb8a21c4aea775ab19a8bb699b5b2395fd62feac7f6f3f848
Jul 24 17:10:57.558423 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.558360    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hts7p_11a91422-52c6-44f1-b193-7f2ed40394ca/serve-healthcheck-canary/0.log"
Jul 24 17:10:57.786997 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.786960    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7" event={"ID":"7a2e3f2b-769a-4187-ba23-ce268bdb7d9b","Type":"ContainerStarted","Data":"1c1e670ffd67360140a8e115dcca7a3c883dda7225bc8da4010dcfa0113a1753"}
Jul 24 17:10:57.787205 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.787041    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7" event={"ID":"7a2e3f2b-769a-4187-ba23-ce268bdb7d9b","Type":"ContainerStarted","Data":"1736a3dd81375fd438041d04caa84c010c4102b1a431bfec3e7d0b9462b822ec"}
Jul 24 17:10:57.788128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.788083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-f26cr" event={"ID":"91eccd70-9a48-48d6-82cf-b634314cb67a","Type":"ContainerStarted","Data":"9a5021ca876b714cb8a21c4aea775ab19a8bb699b5b2395fd62feac7f6f3f848"}
Jul 24 17:10:57.804478 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.804432    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-4qbc7" podStartSLOduration=1.991910227 podStartE2EDuration="3.804418266s" totalImagesPullingTime="1.812508039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:10:57.8037005 +0000 UTC m=+46.774832681" watchObservedRunningTime="2026-07-24 17:10:57.804418266 +0000 UTC m=+46.775550446"
Jul 24 17:10:57.954792 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.954763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:57.954947 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:10:57.954833    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:10:57.955047 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:57.954941    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:10:57.955047 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:57.954957    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:10:57.955047 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:57.955013    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:11:01.954999287 +0000 UTC m=+50.926131445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : configmap "kube-rbac-proxy" not found
Jul 24 17:10:57.955047 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:10:57.955044    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:11:01.955038321 +0000 UTC m=+50.926170480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:11:00.797339 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:00.797299    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-f26cr" event={"ID":"91eccd70-9a48-48d6-82cf-b634314cb67a","Type":"ContainerStarted","Data":"823c4c07273eeca58ac4cbf982a8281de7c48c4328ed20139333c2160ef66fc0"}
Jul 24 17:11:00.813031 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:00.812965    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-f26cr" podStartSLOduration=2.41942506 podStartE2EDuration="4.812947925s" totalImagesPullingTime="2.393522865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:00.812254061 +0000 UTC m=+49.783386242" watchObservedRunningTime="2026-07-24 17:11:00.812947925 +0000 UTC m=+49.784080110"
Jul 24 17:11:01.991251 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:01.991215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:11:01.991661 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:01.991326    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:11:01.991661 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:01.991441    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:11:01.991661 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:01.991516    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls podName:3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e nodeName:}" failed. No retries permitted until 2026-07-24 17:11:09.991496117 +0000 UTC m=+58.962628278 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6c59z" (UID: "3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:11:01.991865 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:01.991841    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:11:02.798433 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:02.798396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:11:02.798609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:02.798459    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:11:02.798609 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:02.798563    2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 17:11:02.798609 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:02.798569    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:11:02.798714 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:02.798649    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls podName:dde92a18-877a-423d-9e1b-ed03e6638e03 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:18.798626706 +0000 UTC m=+67.769758872 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls") pod "dns-default-v8khf" (UID: "dde92a18-877a-423d-9e1b-ed03e6638e03") : secret "dns-default-metrics-tls" not found
Jul 24 17:11:02.798714 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:02.798573    2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d474986c-7zdg2: secret "image-registry-tls" not found
Jul 24 17:11:02.798714 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:02.798708    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls podName:572b70ea-b429-4a02-995a-81eb41af79c8 nodeName:}" failed. No retries permitted until 2026-07-24 17:11:18.798694062 +0000 UTC m=+67.769826222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls") pod "image-registry-5d474986c-7zdg2" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8") : secret "image-registry-tls" not found
Jul 24 17:11:10.049330 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:10.049271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:11:10.051684 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:10.051658    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6c59z\" (UID: \"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e\") " pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:11:10.119970 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:10.119936    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6c59z"
Jul 24 17:11:10.253878 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:10.253845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6c59z"]
Jul 24 17:11:10.257136 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:11:10.257109    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3078dc6e_e8fa_4b3f_af1b_cdf73cfd0c4e.slice/crio-8cdc3d4a4d4e7211895206bb6806b5e74c2c426267eb73e1c71c1a8389be0dc5 WatchSource:0}: Error finding container 8cdc3d4a4d4e7211895206bb6806b5e74c2c426267eb73e1c71c1a8389be0dc5: Status 404 returned error can't find the container with id 8cdc3d4a4d4e7211895206bb6806b5e74c2c426267eb73e1c71c1a8389be0dc5
Jul 24 17:11:10.826207 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:10.826161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6c59z" event={"ID":"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e","Type":"ContainerStarted","Data":"8cdc3d4a4d4e7211895206bb6806b5e74c2c426267eb73e1c71c1a8389be0dc5"}
Jul 24 17:11:12.746513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:12.746476    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zp6nk"
Jul 24 17:11:15.198686 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.198654    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"]
Jul 24 17:11:15.200572 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.200552    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"]
Jul 24 17:11:15.200700 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.200683    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"
Jul 24 17:11:15.202249 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.202233    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"]
Jul 24 17:11:15.202390 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.202373    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.203977 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.203960    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.204089 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.204052    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 17:11:15.204089 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.204066    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 17:11:15.204200 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.204138    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sbpjb\""
Jul 24 17:11:15.204491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.204477    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 17:11:15.205213 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.205191    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 24 17:11:15.205792 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.205765    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 24 17:11:15.206641 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.206622    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 24 17:11:15.213979 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.213962    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 24 17:11:15.214090 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.213962    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 24 17:11:15.215299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.215278    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 24 17:11:15.219718 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.219700    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"]
Jul 24 17:11:15.223173 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.223153    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"]
Jul 24 17:11:15.229475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.229455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"]
Jul 24 17:11:15.289602 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.289789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/65073433-f92e-4a34-9032-6a427187a3fb-klusterlet-config\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.289789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.289789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289720    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-ca\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.289931 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngn5t\" (UniqueName: \"kubernetes.io/projected/65073433-f92e-4a34-9032-6a427187a3fb-kube-api-access-ngn5t\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.289931 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bf5894544-hxr7b\" (UID: \"cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"
Jul 24 17:11:15.289931 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289851    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-hub\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.289931 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65073433-f92e-4a34-9032-6a427187a3fb-tmp\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.290130 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289937    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.290130 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289968    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfq8r\" (UniqueName: \"kubernetes.io/projected/cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8-kube-api-access-cfq8r\") pod \"managed-serviceaccount-addon-agent-6bf5894544-hxr7b\" (UID: \"cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"
Jul 24 17:11:15.290130 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.289999    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5fvw\" (UniqueName: \"kubernetes.io/projected/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-kube-api-access-k5fvw\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.390588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.390506    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.390588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.390542    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-ca\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.390588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.390573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngn5t\" (UniqueName: \"kubernetes.io/projected/65073433-f92e-4a34-9032-6a427187a3fb-kube-api-access-ngn5t\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.390941 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.390709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bf5894544-hxr7b\" (UID: \"cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"
Jul 24 17:11:15.390941 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.390760    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-hub\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.390941 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.390822    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65073433-f92e-4a34-9032-6a427187a3fb-tmp\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.391243 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.391221    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65073433-f92e-4a34-9032-6a427187a3fb-tmp\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.391305 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.391265    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.391359 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.391318    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfq8r\" (UniqueName: \"kubernetes.io/projected/cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8-kube-api-access-cfq8r\") pod \"managed-serviceaccount-addon-agent-6bf5894544-hxr7b\" (UID: \"cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"
Jul 24 17:11:15.391815 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.391789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5fvw\" (UniqueName: \"kubernetes.io/projected/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-kube-api-access-k5fvw\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.391917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.391854    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.391978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.391927    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/65073433-f92e-4a34-9032-6a427187a3fb-klusterlet-config\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.392722 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.392694    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.393953 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.393837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bf5894544-hxr7b\" (UID: \"cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"
Jul 24 17:11:15.393953 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.393878    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.393953 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.393913    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-hub\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.394165 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.394056    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.394440 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.394421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-ca\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.394586 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.394572    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/65073433-f92e-4a34-9032-6a427187a3fb-klusterlet-config\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.399551 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.399524    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngn5t\" (UniqueName: \"kubernetes.io/projected/65073433-f92e-4a34-9032-6a427187a3fb-kube-api-access-ngn5t\") pod \"klusterlet-addon-workmgr-6956bfddc-w72tl\" (UID: \"65073433-f92e-4a34-9032-6a427187a3fb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.400544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.400526    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfq8r\" (UniqueName: \"kubernetes.io/projected/cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8-kube-api-access-cfq8r\") pod \"managed-serviceaccount-addon-agent-6bf5894544-hxr7b\" (UID: \"cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"
Jul 24 17:11:15.400641 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.400617    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5fvw\" (UniqueName: \"kubernetes.io/projected/29cc93df-7cb7-451b-8a42-69d7b9c9b79a-kube-api-access-k5fvw\") pod \"cluster-proxy-proxy-agent-566968cd9f-mfd56\" (UID: \"29cc93df-7cb7-451b-8a42-69d7b9c9b79a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.525854 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.525822    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"
Jul 24 17:11:15.532795 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.532768    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:15.537694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.537490    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"
Jul 24 17:11:15.708806 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.708775    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"]
Jul 24 17:11:15.721914 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.721885    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56"]
Jul 24 17:11:15.725702 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:11:15.725668    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29cc93df_7cb7_451b_8a42_69d7b9c9b79a.slice/crio-6f1e43d14719bd6c7b8d7b5e600df1991a38bcd44857f1fa32fb83807369c723 WatchSource:0}: Error finding container 6f1e43d14719bd6c7b8d7b5e600df1991a38bcd44857f1fa32fb83807369c723: Status 404 returned error can't find the container with id 6f1e43d14719bd6c7b8d7b5e600df1991a38bcd44857f1fa32fb83807369c723
Jul 24 17:11:15.742954 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.742930    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b"]
Jul 24 17:11:15.746315 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:11:15.746284    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc8dc2bd_ce87_443a_9b87_1c42e9b9cca8.slice/crio-5787c79a5bf9a2f620f07c735c34fc6c343763901df31562fd38e829e29b8fbc WatchSource:0}: Error finding container 5787c79a5bf9a2f620f07c735c34fc6c343763901df31562fd38e829e29b8fbc: Status 404 returned error can't find the container with id 5787c79a5bf9a2f620f07c735c34fc6c343763901df31562fd38e829e29b8fbc
Jul 24 17:11:15.846910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.846867    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6c59z" event={"ID":"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e","Type":"ContainerStarted","Data":"692f70bcadbe3e292e6643c0bafbea469903c28ae73db02526cf7f316e6203c7"}
Jul 24 17:11:15.848459 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.848430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56" event={"ID":"29cc93df-7cb7-451b-8a42-69d7b9c9b79a","Type":"ContainerStarted","Data":"6f1e43d14719bd6c7b8d7b5e600df1991a38bcd44857f1fa32fb83807369c723"}
Jul 24 17:11:15.849906 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.849879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl" event={"ID":"65073433-f92e-4a34-9032-6a427187a3fb","Type":"ContainerStarted","Data":"b84f036c0c3ff7f50b4e26686c4cb834095f560f18327033a6f623ac55d2f676"}
Jul 24 17:11:15.851144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:15.851123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b" event={"ID":"cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8","Type":"ContainerStarted","Data":"5787c79a5bf9a2f620f07c735c34fc6c343763901df31562fd38e829e29b8fbc"}
Jul 24 17:11:17.313690 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.313228    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:11:17.313690 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.313282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:11:17.316100 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.316074    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:11:17.317296 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.317111    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:11:17.337706 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.337667    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/573221db-ed4c-48f8-8267-b5eeab52090a-metrics-certs\") pod \"network-metrics-daemon-8qx5c\" (UID: \"573221db-ed4c-48f8-8267-b5eeab52090a\") " pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:11:17.339876 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.339844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8d915ffa-99b0-4ff7-abc6-ca4c0df718bb-original-pull-secret\") pod \"global-pull-secret-syncer-ptxkr\" (UID: \"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb\") " pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:11:17.414337 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.414222    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:11:17.416895 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.416857    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 17:11:17.427363 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.427331    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 17:11:17.442440 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.442385    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmhtj\" (UniqueName: \"kubernetes.io/projected/d3a36ac0-84b7-4717-be5b-9bca1c9de683-kube-api-access-hmhtj\") pod \"network-check-target-th2mm\" (UID: \"d3a36ac0-84b7-4717-be5b-9bca1c9de683\") " pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:11:17.509766 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.509737    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ptxkr"
Jul 24 17:11:17.517578 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.517550    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gkgpl\""
Jul 24 17:11:17.522213 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.522008    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-56frs\""
Jul 24 17:11:17.525567 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.525485    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:11:17.529898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:17.529832    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8qx5c"
Jul 24 17:11:18.828566 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:18.828527    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:11:18.829036 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:18.828583    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:11:18.832165 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:18.832141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/dde92a18-877a-423d-9e1b-ed03e6638e03-metrics-tls\") pod \"dns-default-v8khf\" (UID: \"dde92a18-877a-423d-9e1b-ed03e6638e03\") " pod="openshift-dns/dns-default-v8khf"
Jul 24 17:11:18.832296 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:18.832226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"image-registry-5d474986c-7zdg2\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") " pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:11:19.089178 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:19.089094    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g65mg\""
Jul 24 17:11:19.094116 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:19.094081    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cpjrs\""
Jul 24 17:11:19.096661 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:19.096465    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:11:19.101784 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:19.101605    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v8khf"
Jul 24 17:11:23.190170 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.189870    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-th2mm"]
Jul 24 17:11:23.214647 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.214576    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ptxkr"]
Jul 24 17:11:23.230598 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.230536    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d474986c-7zdg2"]
Jul 24 17:11:23.445475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.445453    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v8khf"]
Jul 24 17:11:23.448552 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:11:23.448505    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde92a18_877a_423d_9e1b_ed03e6638e03.slice/crio-0e26a8060c7fece62a27a6d426b3738c904815af57f4df4bd8589ca5985d1714 WatchSource:0}: Error finding container 0e26a8060c7fece62a27a6d426b3738c904815af57f4df4bd8589ca5985d1714: Status 404 returned error can't find the container with id 0e26a8060c7fece62a27a6d426b3738c904815af57f4df4bd8589ca5985d1714
Jul 24 17:11:23.449968 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.449948    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8qx5c"]
Jul 24 17:11:23.453009 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:11:23.452990    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573221db_ed4c_48f8_8267_b5eeab52090a.slice/crio-6674b3c6c07617d702c48355b5e27614fe7eb923a51322141fe764a5266d3270 WatchSource:0}: Error finding container 6674b3c6c07617d702c48355b5e27614fe7eb923a51322141fe764a5266d3270: Status 404 returned error can't find the container with id 6674b3c6c07617d702c48355b5e27614fe7eb923a51322141fe764a5266d3270
Jul 24 17:11:23.892908 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.892741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6c59z" event={"ID":"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e","Type":"ContainerStarted","Data":"4e24585fcea565e83f72cd1fcf1f2c2084e79ae7ca657f3c0c919c60ab5d92e8"}
Jul 24 17:11:23.896086 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.896056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56" event={"ID":"29cc93df-7cb7-451b-8a42-69d7b9c9b79a","Type":"ContainerStarted","Data":"ee7b65e1f1a66f32e011a18c039e403102010aaa93c8b51e39536cddaee06ece"}
Jul 24 17:11:23.896220 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.896096    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56" event={"ID":"29cc93df-7cb7-451b-8a42-69d7b9c9b79a","Type":"ContainerStarted","Data":"5c6a3bebaf62280a635bd2afdeb832e7138437e646617878e753053254a16814"}
Jul 24 17:11:23.896220 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.896113    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56" event={"ID":"29cc93df-7cb7-451b-8a42-69d7b9c9b79a","Type":"ContainerStarted","Data":"e4abcd74ef5bfc152fa7c21e0367cb2b6182df2b4296e001fb53fef748ba778f"}
Jul 24 17:11:23.915041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.911160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d474986c-7zdg2" event={"ID":"572b70ea-b429-4a02-995a-81eb41af79c8","Type":"ContainerStarted","Data":"ca7d238c25500f3ccd818a02ad854fc2ca8c3eaae0e8317e0c646f132b528fbf"}
Jul 24 17:11:23.915041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.911751    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d474986c-7zdg2" event={"ID":"572b70ea-b429-4a02-995a-81eb41af79c8","Type":"ContainerStarted","Data":"6918e7ccaac34b3ed1a313758c9c6bf055007ed3d1f94bade86bacb770e4d3ae"}
Jul 24 17:11:23.915041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.911845    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:11:23.915041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.914061    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl" event={"ID":"65073433-f92e-4a34-9032-6a427187a3fb","Type":"ContainerStarted","Data":"31efb085cbca939a8bdaef9a45b32445fa1b78d16124401f2ef41534488c07f0"}
Jul 24 17:11:23.915041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.914712    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:23.917237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.917209    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl"
Jul 24 17:11:23.920475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.919483    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8khf" event={"ID":"dde92a18-877a-423d-9e1b-ed03e6638e03","Type":"ContainerStarted","Data":"0e26a8060c7fece62a27a6d426b3738c904815af57f4df4bd8589ca5985d1714"}
Jul 24 17:11:23.921108 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.921067    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-566968cd9f-mfd56" podStartSLOduration=1.644413733 podStartE2EDuration="8.92104997s" totalImagesPullingTime="7.276636237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:23.918402622 +0000 UTC m=+72.889534803" watchObservedRunningTime="2026-07-24 17:11:23.92104997 +0000 UTC m=+72.892182153"
Jul 24 17:11:23.924109 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.924047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8qx5c" event={"ID":"573221db-ed4c-48f8-8267-b5eeab52090a","Type":"ContainerStarted","Data":"6674b3c6c07617d702c48355b5e27614fe7eb923a51322141fe764a5266d3270"}
Jul 24 17:11:23.926041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.925965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-th2mm" event={"ID":"d3a36ac0-84b7-4717-be5b-9bca1c9de683","Type":"ContainerStarted","Data":"be980bd710fa7ee7e9595238330ba681a2478c69cfe21b6f48540ee5562476f0"}
Jul 24 17:11:23.928233 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.927730    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b" event={"ID":"cc8dc2bd-ce87-443a-9b87-1c42e9b9cca8","Type":"ContainerStarted","Data":"6d1bfa149554cfa559b9732926aee22b68810597879e69dc4a7b671aef9e6022"}
Jul 24 17:11:23.929496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.929445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ptxkr" event={"ID":"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb","Type":"ContainerStarted","Data":"76c217ffd168ffbdadb5bfc2903ba8b84ed181ecc0e86063e8ee3c9c9d9fe4c7"}
Jul 24 17:11:23.942146 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.941923    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d474986c-7zdg2" podStartSLOduration=63.941905776 podStartE2EDuration="1m3.941905776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:23.940738849 +0000 UTC m=+72.911871057" watchObservedRunningTime="2026-07-24 17:11:23.941905776 +0000 UTC m=+72.913037957"
Jul 24 17:11:23.979010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.977685    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6956bfddc-w72tl" podStartSLOduration=1.69541656 podStartE2EDuration="8.97766691s" totalImagesPullingTime="7.28225035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:23.959845485 +0000 UTC m=+72.930977668" watchObservedRunningTime="2026-07-24 17:11:23.97766691 +0000 UTC m=+72.948799093"
Jul 24 17:11:23.979010 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:23.978013    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bf5894544-hxr7b" podStartSLOduration=1.746053555 podStartE2EDuration="8.977999311s" totalImagesPullingTime="7.231945756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:23.977655421 +0000 UTC m=+72.948787602" watchObservedRunningTime="2026-07-24 17:11:23.977999311 +0000 UTC m=+72.949131493"
Jul 24 17:11:25.945377 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:25.945346    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8qx5c" event={"ID":"573221db-ed4c-48f8-8267-b5eeab52090a","Type":"ContainerStarted","Data":"957a980aa94eb35355fcef7bdf9a67d86395c9eb4f9418909a9079b95c5324a7"}
Jul 24 17:11:25.945801 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:25.945393    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8qx5c" event={"ID":"573221db-ed4c-48f8-8267-b5eeab52090a","Type":"ContainerStarted","Data":"1bfac09e53e2e836ac0cb47577e355f6439900f18f4e61590186d199a171c077"}
Jul 24 17:11:25.964878 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:25.964381    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8qx5c" podStartSLOduration=72.580027985 podStartE2EDuration="1m13.964360919s" totalImagesPullingTime="1.384332934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:25.963251431 +0000 UTC m=+74.934383616" watchObservedRunningTime="2026-07-24 17:11:25.964360919 +0000 UTC m=+74.935493105"
Jul 24 17:11:30.965144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.965106    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ptxkr" event={"ID":"8d915ffa-99b0-4ff7-abc6-ca4c0df718bb","Type":"ContainerStarted","Data":"477b2db07c1c5796cd5ed142ec334482a594970a2d749578704cff224cf2129e"}
Jul 24 17:11:30.967323 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.967302    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6c59z" event={"ID":"3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e","Type":"ContainerStarted","Data":"f6ddb96199703772af51f8798268e0f6eac6ae7d43c03dcf8ba7c8f78f7a3edf"}
Jul 24 17:11:30.969062 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.969035    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8khf" event={"ID":"dde92a18-877a-423d-9e1b-ed03e6638e03","Type":"ContainerStarted","Data":"130caa6b7cc7356d0c9aae4a70fd3edb272d918af7118cf7197917a9c73e9dc7"}
Jul 24 17:11:30.969164 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.969083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v8khf" event={"ID":"dde92a18-877a-423d-9e1b-ed03e6638e03","Type":"ContainerStarted","Data":"3153087981b327c869b5c33b4b1ba25a50c08d7682a9160a0e7e4f19e4dbc04b"}
Jul 24 17:11:30.969164 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.969104    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v8khf"
Jul 24 17:11:30.970439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.970418    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-th2mm" event={"ID":"d3a36ac0-84b7-4717-be5b-9bca1c9de683","Type":"ContainerStarted","Data":"efe37eeb98ca723dee00ca5cb3ff7732810889e7c9ee618613a94f873dbad623"}
Jul 24 17:11:30.970548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.970538    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:11:30.981692 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.981647    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ptxkr" podStartSLOduration=72.159189486 podStartE2EDuration="1m18.981633623s" totalImagesPullingTime="6.822444137s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:30.980878641 +0000 UTC m=+79.952010822" watchObservedRunningTime="2026-07-24 17:11:30.981633623 +0000 UTC m=+79.952765803"
Jul 24 17:11:30.995244 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:30.995206    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-th2mm" podStartSLOduration=72.140813223 podStartE2EDuration="1m18.995193334s" totalImagesPullingTime="6.854380111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:30.994541705 +0000 UTC m=+79.965673886" watchObservedRunningTime="2026-07-24 17:11:30.995193334 +0000 UTC m=+79.966325515"
Jul 24 17:11:31.010660 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:31.010620    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v8khf" podStartSLOduration=38.41309912 podStartE2EDuration="45.010605022s" totalImagesPullingTime="6.597505902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:10:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:31.010094122 +0000 UTC m=+79.981226302" watchObservedRunningTime="2026-07-24 17:11:31.010605022 +0000 UTC m=+79.981737185"
Jul 24 17:11:31.028339 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:31.028308    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6c59z" podStartSLOduration=17.427540794 podStartE2EDuration="37.028296647s" totalImagesPullingTime="19.600755853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:10:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:31.02796618 +0000 UTC m=+79.999098361" watchObservedRunningTime="2026-07-24 17:11:31.028296647 +0000 UTC m=+79.999428822"
Jul 24 17:11:37.540699 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:37.540576    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d474986c-7zdg2"]
Jul 24 17:11:38.809398 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.809354    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-44cwx"]
Jul 24 17:11:38.814079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.814056    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.817486 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.817469    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 17:11:38.818540 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.818522    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 17:11:38.818746 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.818727    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 17:11:38.818812 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.818733    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 17:11:38.818812 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.818776    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 17:11:38.819191 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.819171    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kgtc8\""
Jul 24 17:11:38.819292 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.818657    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 17:11:38.888851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.888816    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.889072 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.888873    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-wtmp\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.889072 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.888956    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-sys\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.889072 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.888998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-root\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.889072 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.889062    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.889287 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.889122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-tls\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.889287 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.889156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-textfile\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.889287 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.889195    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-metrics-client-ca\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.889287 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.889220    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn5dp\" (UniqueName: \"kubernetes.io/projected/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-kube-api-access-dn5dp\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990367 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990330    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-sys\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990378    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-root\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-root\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990456    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-sys\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-tls\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990824 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-textfile\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990824 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-metrics-client-ca\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990824 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990656    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn5dp\" (UniqueName: \"kubernetes.io/projected/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-kube-api-access-dn5dp\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990824 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990824 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:38.990701    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 17:11:38.990824 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990742    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-wtmp\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.990824 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:11:38.990760    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-tls podName:dad5a54f-a819-4aa0-b5a0-97f614e52a9a nodeName:}" failed. No retries permitted until 2026-07-24 17:11:39.490741016 +0000 UTC m=+88.461873178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-tls") pod "node-exporter-44cwx" (UID: "dad5a54f-a819-4aa0-b5a0-97f614e52a9a") : secret "node-exporter-tls" not found
Jul 24 17:11:38.991196 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.990898    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-wtmp\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.991246 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.991204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-textfile\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.991770 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.991744    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-metrics-client-ca\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.991826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.991770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:38.994652 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.994628    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:39.001046 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:38.999690    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn5dp\" (UniqueName: \"kubernetes.io/projected/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-kube-api-access-dn5dp\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:39.494752 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:39.494717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-tls\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:39.497581 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:39.497548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dad5a54f-a819-4aa0-b5a0-97f614e52a9a-node-exporter-tls\") pod \"node-exporter-44cwx\" (UID: \"dad5a54f-a819-4aa0-b5a0-97f614e52a9a\") " pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:39.725737 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:39.725711    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-44cwx"
Jul 24 17:11:39.738781 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:11:39.738752    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddad5a54f_a819_4aa0_b5a0_97f614e52a9a.slice/crio-5b2eec4b23846fa25dc0a3e994db2ca39e6f6bd881257c3a07bfab3311a2efc9 WatchSource:0}: Error finding container 5b2eec4b23846fa25dc0a3e994db2ca39e6f6bd881257c3a07bfab3311a2efc9: Status 404 returned error can't find the container with id 5b2eec4b23846fa25dc0a3e994db2ca39e6f6bd881257c3a07bfab3311a2efc9
Jul 24 17:11:40.001689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:40.001658    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44cwx" event={"ID":"dad5a54f-a819-4aa0-b5a0-97f614e52a9a","Type":"ContainerStarted","Data":"5b2eec4b23846fa25dc0a3e994db2ca39e6f6bd881257c3a07bfab3311a2efc9"}
Jul 24 17:11:40.975952 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:40.975925    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v8khf"
Jul 24 17:11:41.007114 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:41.007079    2562 generic.go:356] "Generic (PLEG): container finished" podID="dad5a54f-a819-4aa0-b5a0-97f614e52a9a" containerID="bc1fc7f5e85e2ddaba0cff22e977d74b87a482d3561f761e15747c93790aa16b" exitCode=0
Jul 24 17:11:41.007499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:41.007177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44cwx" event={"ID":"dad5a54f-a819-4aa0-b5a0-97f614e52a9a","Type":"ContainerDied","Data":"bc1fc7f5e85e2ddaba0cff22e977d74b87a482d3561f761e15747c93790aa16b"}
Jul 24 17:11:42.012889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:42.012857    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44cwx" event={"ID":"dad5a54f-a819-4aa0-b5a0-97f614e52a9a","Type":"ContainerStarted","Data":"4d086ee83f0da653eb7e07dce88e4c90803b88b373635035b001e23340ff1e77"}
Jul 24 17:11:42.013286 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:42.012909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-44cwx" event={"ID":"dad5a54f-a819-4aa0-b5a0-97f614e52a9a","Type":"ContainerStarted","Data":"fad03fe60203c32d3b9c1632d5da32156e3d7e27c8865dd6b841570a4be9d6fd"}
Jul 24 17:11:42.032680 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:42.032630    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-44cwx" podStartSLOduration=3.105990675 podStartE2EDuration="4.032614732s" totalImagesPullingTime="856.388401ms" totalInitContainerRuntime="70.235656ms" isStatefulPod=true podCreationTimestamp="2026-07-24 17:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:11:42.030916171 +0000 UTC m=+91.002048352" watchObservedRunningTime="2026-07-24 17:11:42.032614732 +0000 UTC m=+91.003746914"
Jul 24 17:11:47.545879 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:11:47.545845    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:12:01.977152 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:01.977120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-th2mm"
Jul 24 17:12:02.559892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:02.559827    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5d474986c-7zdg2" podUID="572b70ea-b429-4a02-995a-81eb41af79c8" containerName="registry" containerID="cri-o://ca7d238c25500f3ccd818a02ad854fc2ca8c3eaae0e8317e0c646f132b528fbf" gracePeriod=30
Jul 24 17:12:03.084472 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.084435    2562 generic.go:356] "Generic (PLEG): container finished" podID="572b70ea-b429-4a02-995a-81eb41af79c8" containerID="ca7d238c25500f3ccd818a02ad854fc2ca8c3eaae0e8317e0c646f132b528fbf" exitCode=0
Jul 24 17:12:03.084877 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.084506    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d474986c-7zdg2" event={"ID":"572b70ea-b429-4a02-995a-81eb41af79c8","Type":"ContainerDied","Data":"ca7d238c25500f3ccd818a02ad854fc2ca8c3eaae0e8317e0c646f132b528fbf"}
Jul 24 17:12:03.326252 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.326220    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:12:03.461074 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461046    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-registry-certificates\") pod \"572b70ea-b429-4a02-995a-81eb41af79c8\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") "
Jul 24 17:12:03.461074 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461075    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-bound-sa-token\") pod \"572b70ea-b429-4a02-995a-81eb41af79c8\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") "
Jul 24 17:12:03.461310 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461111    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-installation-pull-secrets\") pod \"572b70ea-b429-4a02-995a-81eb41af79c8\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") "
Jul 24 17:12:03.461310 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461140    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-image-registry-private-configuration\") pod \"572b70ea-b429-4a02-995a-81eb41af79c8\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") "
Jul 24 17:12:03.461310 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461156    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") pod \"572b70ea-b429-4a02-995a-81eb41af79c8\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") "
Jul 24 17:12:03.461310 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461283    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk6n5\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-kube-api-access-qk6n5\") pod \"572b70ea-b429-4a02-995a-81eb41af79c8\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") "
Jul 24 17:12:03.461468 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461333    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-trusted-ca\") pod \"572b70ea-b429-4a02-995a-81eb41af79c8\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") "
Jul 24 17:12:03.461468 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461361    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572b70ea-b429-4a02-995a-81eb41af79c8-ca-trust-extracted\") pod \"572b70ea-b429-4a02-995a-81eb41af79c8\" (UID: \"572b70ea-b429-4a02-995a-81eb41af79c8\") "
Jul 24 17:12:03.461598 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461574    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "572b70ea-b429-4a02-995a-81eb41af79c8" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:12:03.461913 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.461772    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "572b70ea-b429-4a02-995a-81eb41af79c8" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:12:03.464124 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.464090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "572b70ea-b429-4a02-995a-81eb41af79c8" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:12:03.464272 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.464131    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "572b70ea-b429-4a02-995a-81eb41af79c8" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:12:03.464272 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.464186    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "572b70ea-b429-4a02-995a-81eb41af79c8" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:12:03.464272 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.464248    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "572b70ea-b429-4a02-995a-81eb41af79c8" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:12:03.464476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.464316    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-kube-api-access-qk6n5" (OuterVolumeSpecName: "kube-api-access-qk6n5") pod "572b70ea-b429-4a02-995a-81eb41af79c8" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8"). InnerVolumeSpecName "kube-api-access-qk6n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:12:03.469714 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.469689    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/572b70ea-b429-4a02-995a-81eb41af79c8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "572b70ea-b429-4a02-995a-81eb41af79c8" (UID: "572b70ea-b429-4a02-995a-81eb41af79c8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:12:03.562368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.562328    2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-installation-pull-secrets\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:12:03.562368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.562356    2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/572b70ea-b429-4a02-995a-81eb41af79c8-image-registry-private-configuration\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:12:03.562368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.562367    2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-registry-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:12:03.562368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.562376    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk6n5\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-kube-api-access-qk6n5\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:12:03.562694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.562385    2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-trusted-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:12:03.562694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.562394    2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/572b70ea-b429-4a02-995a-81eb41af79c8-ca-trust-extracted\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:12:03.562694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.562401    2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/572b70ea-b429-4a02-995a-81eb41af79c8-registry-certificates\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:12:03.562694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:03.562409    2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/572b70ea-b429-4a02-995a-81eb41af79c8-bound-sa-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:12:04.088685 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:04.088656    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d474986c-7zdg2"
Jul 24 17:12:04.089133 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:04.088654    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d474986c-7zdg2" event={"ID":"572b70ea-b429-4a02-995a-81eb41af79c8","Type":"ContainerDied","Data":"6918e7ccaac34b3ed1a313758c9c6bf055007ed3d1f94bade86bacb770e4d3ae"}
Jul 24 17:12:04.089133 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:04.088784    2562 scope.go:117] "RemoveContainer" containerID="ca7d238c25500f3ccd818a02ad854fc2ca8c3eaae0e8317e0c646f132b528fbf"
Jul 24 17:12:04.108895 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:04.108863    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d474986c-7zdg2"]
Jul 24 17:12:04.114793 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:04.114765    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5d474986c-7zdg2"]
Jul 24 17:12:05.503670 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:12:05.503631    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="572b70ea-b429-4a02-995a-81eb41af79c8" path="/var/lib/kubelet/pods/572b70ea-b429-4a02-995a-81eb41af79c8/volumes"
Jul 24 17:15:11.390428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:11.390399    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:15:11.392051 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:11.392031    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:15:11.397096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:11.397078    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 17:15:19.350203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.350170    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"]
Jul 24 17:15:19.351506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.350401    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="572b70ea-b429-4a02-995a-81eb41af79c8" containerName="registry"
Jul 24 17:15:19.351506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.350411    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="572b70ea-b429-4a02-995a-81eb41af79c8" containerName="registry"
Jul 24 17:15:19.351506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.350454    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="572b70ea-b429-4a02-995a-81eb41af79c8" containerName="registry"
Jul 24 17:15:19.352151 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.352138    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.356674 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.356654    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 17:15:19.356835 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.356817    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4tjsr\""
Jul 24 17:15:19.356913 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.356898    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 17:15:19.357186 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.357174    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 17:15:19.367903 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.367883    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 17:15:19.381522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.381491    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.381522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.381521    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-tmp\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.381705 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.381540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-manifests\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.381705 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.381616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v75xs\" (UniqueName: \"kubernetes.io/projected/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-kube-api-access-v75xs\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.381705 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.381646    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.400444 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.400411    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"]
Jul 24 17:15:19.482655 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.482617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v75xs\" (UniqueName: \"kubernetes.io/projected/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-kube-api-access-v75xs\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.482849 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.482664    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.482849 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.482710    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.482849 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.482731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-tmp\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.482849 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.482756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-manifests\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.483352 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.483324    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-manifests\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.485288 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.485268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-tmp\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.485388 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.485340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-webhook-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.485500 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.485480    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.491252 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.491228    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v75xs\" (UniqueName: \"kubernetes.io/projected/5e0c6ff1-ad9b-47e1-947d-57762a4704d9-kube-api-access-v75xs\") pod \"opendatahub-operator-controller-manager-5bcd74455c-zt55x\" (UID: \"5e0c6ff1-ad9b-47e1-947d-57762a4704d9\") " pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.662511 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.662414    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:19.796348 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.796206    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"]
Jul 24 17:15:19.799270 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:15:19.799245    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0c6ff1_ad9b_47e1_947d_57762a4704d9.slice/crio-e76eae2264059167a84eb65397bcf42c0719e695f47420f7efa5fab73302f3bf WatchSource:0}: Error finding container e76eae2264059167a84eb65397bcf42c0719e695f47420f7efa5fab73302f3bf: Status 404 returned error can't find the container with id e76eae2264059167a84eb65397bcf42c0719e695f47420f7efa5fab73302f3bf
Jul 24 17:15:19.801584 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:19.801568    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:15:20.731233 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:20.731194    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x" event={"ID":"5e0c6ff1-ad9b-47e1-947d-57762a4704d9","Type":"ContainerStarted","Data":"e76eae2264059167a84eb65397bcf42c0719e695f47420f7efa5fab73302f3bf"}
Jul 24 17:15:22.739513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:22.739471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x" event={"ID":"5e0c6ff1-ad9b-47e1-947d-57762a4704d9","Type":"ContainerStarted","Data":"412dba531d4237222d95b8b5eb05dc18aa74abe891f03f5b3d059ad4b4ed9d4a"}
Jul 24 17:15:24.747279 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:24.747242    2562 generic.go:356] "Generic (PLEG): container finished" podID="5e0c6ff1-ad9b-47e1-947d-57762a4704d9" containerID="412dba531d4237222d95b8b5eb05dc18aa74abe891f03f5b3d059ad4b4ed9d4a" exitCode=0
Jul 24 17:15:24.747681 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:24.747300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x" event={"ID":"5e0c6ff1-ad9b-47e1-947d-57762a4704d9","Type":"ContainerDied","Data":"412dba531d4237222d95b8b5eb05dc18aa74abe891f03f5b3d059ad4b4ed9d4a"}
Jul 24 17:15:25.751909 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:25.751874    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x" event={"ID":"5e0c6ff1-ad9b-47e1-947d-57762a4704d9","Type":"ContainerStarted","Data":"1318ab0774f0049df4d80bc9352d4955078a7a99490789736bfd731e6da8e460"}
Jul 24 17:15:25.752380 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:25.752094    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:25.774237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:25.774191    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x" podStartSLOduration=2.47210675 podStartE2EDuration="6.774177045s" totalImagesPullingTime="3.236039569s" totalInitContainerRuntime="1.066030726s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:25.772656484 +0000 UTC m=+314.743788664" watchObservedRunningTime="2026-07-24 17:15:25.774177045 +0000 UTC m=+314.745309226"
Jul 24 17:15:36.757904 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:36.757874    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5bcd74455c-zt55x"
Jul 24 17:15:37.620775 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.620741    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"]
Jul 24 17:15:37.626236 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.626221    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.629394 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.629372    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:15:37.630600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.630582    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 17:15:37.630600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.630590    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6snd8\""
Jul 24 17:15:37.630600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.630602    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 17:15:37.630808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.630689    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 17:15:37.630808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.630695    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 17:15:37.634998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.634981    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"]
Jul 24 17:15:37.717197 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.717161    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-cert\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.717197 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.717200    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-metrics-cert\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.717398 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.717224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vqmj\" (UniqueName: \"kubernetes.io/projected/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-kube-api-access-4vqmj\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.717398 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.717301    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-manager-config\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.818217 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.818177    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-cert\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.818217 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.818221    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-metrics-cert\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.818633 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.818241    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqmj\" (UniqueName: \"kubernetes.io/projected/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-kube-api-access-4vqmj\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.818633 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.818264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-manager-config\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.818876 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.818858    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-manager-config\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.820767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.820741    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-cert\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.820902 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.820879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-metrics-cert\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.827232 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.827212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vqmj\" (UniqueName: \"kubernetes.io/projected/c34cf630-e0de-4ae5-ba20-d9767dcb39c3-kube-api-access-4vqmj\") pod \"lws-controller-manager-57c7c45669-zx7rw\" (UID: \"c34cf630-e0de-4ae5-ba20-d9767dcb39c3\") " pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:37.935980 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:37.935896    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:38.070242 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:38.070213    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"]
Jul 24 17:15:38.073359 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:15:38.073330    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc34cf630_e0de_4ae5_ba20_d9767dcb39c3.slice/crio-7ced6b911b09a3cf273bb77c3608aaabe7b729a6e13442947247ca9dc3e157f2 WatchSource:0}: Error finding container 7ced6b911b09a3cf273bb77c3608aaabe7b729a6e13442947247ca9dc3e157f2: Status 404 returned error can't find the container with id 7ced6b911b09a3cf273bb77c3608aaabe7b729a6e13442947247ca9dc3e157f2
Jul 24 17:15:38.797095 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:38.797003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw" event={"ID":"c34cf630-e0de-4ae5-ba20-d9767dcb39c3","Type":"ContainerStarted","Data":"7ced6b911b09a3cf273bb77c3608aaabe7b729a6e13442947247ca9dc3e157f2"}
Jul 24 17:15:40.806044 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:40.805932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw" event={"ID":"c34cf630-e0de-4ae5-ba20-d9767dcb39c3","Type":"ContainerStarted","Data":"3e04bb32a520fe6ea04c41ec4b397a8bf75034b39a965e0548f37ffefc83da7a"}
Jul 24 17:15:40.806455 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:40.806061    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:40.822266 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:40.822188    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw" podStartSLOduration=1.44369014 podStartE2EDuration="3.822168952s" totalImagesPullingTime="2.378478812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:40.821936097 +0000 UTC m=+329.793068278" watchObservedRunningTime="2026-07-24 17:15:40.822168952 +0000 UTC m=+329.793301135"
Jul 24 17:15:46.451881 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.451850    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"]
Jul 24 17:15:46.464753 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.464735    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.467603 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.467575    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 17:15:46.467728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.467579    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"]
Jul 24 17:15:46.467728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.467579    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 17:15:46.467728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.467707    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 17:15:46.467728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.467650    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2z5pt\""
Jul 24 17:15:46.588660 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588623    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.588660 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588664    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.588910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588689    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfkhg\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-kube-api-access-qfkhg\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.588910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.588910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.588910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588813    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.589128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.589128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588948    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.589128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.588980    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689685 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689576    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689685 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689628    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689685 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfkhg\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-kube-api-access-qfkhg\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689847 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689847 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689738    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689847 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689773    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.689847 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689804    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.690083 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689898    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.690083 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.689965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.690083 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.690005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.690244 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.690115    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.690303 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.690273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.692159 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.692141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.692419 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.692390    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.701234 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.701211    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.701502 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.701483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfkhg\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-kube-api-access-qfkhg\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcngggp\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.776430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.776337    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:46.799437 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.799404    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"]
Jul 24 17:15:46.802640 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.802623    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.814044 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.814002    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"]
Jul 24 17:15:46.891505 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.891505 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891512    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a9555d6-bef2-49cd-8a69-ee304a259828-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.891731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.891731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.891731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891686    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.891731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891703    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.891903 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgcjz\" (UniqueName: \"kubernetes.io/projected/1a9555d6-bef2-49cd-8a69-ee304a259828-kube-api-access-dgcjz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.891903 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891764    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.891903 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.891804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.913269 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.913240    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"]
Jul 24 17:15:46.916475 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:15:46.916450    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fc5d37e_9881_47f9_ac6d_c723fe4cb208.slice/crio-b796b111a394b69c2cef46cd1b62886a87697f731075b73642352d7a017302f5 WatchSource:0}: Error finding container b796b111a394b69c2cef46cd1b62886a87697f731075b73642352d7a017302f5: Status 404 returned error can't find the container with id b796b111a394b69c2cef46cd1b62886a87697f731075b73642352d7a017302f5
Jul 24 17:15:46.992749 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.992749 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992745    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a9555d6-bef2-49cd-8a69-ee304a259828-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.992998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.992998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992783    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.992998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992820    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.992998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992848    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.992998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992875    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgcjz\" (UniqueName: \"kubernetes.io/projected/1a9555d6-bef2-49cd-8a69-ee304a259828-kube-api-access-dgcjz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.992998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992903    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.992998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.992935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.993392 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.993272    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.993459 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.993405    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.993507 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.993483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.993551 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.993537    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a9555d6-bef2-49cd-8a69-ee304a259828-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.993713 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.993613    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.995257 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.995237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:46.995497 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:46.995478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:47.000990 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:47.000968    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a9555d6-bef2-49cd-8a69-ee304a259828-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:47.001651 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:47.001632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgcjz\" (UniqueName: \"kubernetes.io/projected/1a9555d6-bef2-49cd-8a69-ee304a259828-kube-api-access-dgcjz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77\" (UID: \"1a9555d6-bef2-49cd-8a69-ee304a259828\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:47.112903 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:47.112824    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:47.252099 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:47.252047    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"]
Jul 24 17:15:47.255843 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:15:47.255813    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9555d6_bef2_49cd_8a69_ee304a259828.slice/crio-aadea36161de58d7ab11272861b82e2fe34aae76f05626b0795276cf97cc34b8 WatchSource:0}: Error finding container aadea36161de58d7ab11272861b82e2fe34aae76f05626b0795276cf97cc34b8: Status 404 returned error can't find the container with id aadea36161de58d7ab11272861b82e2fe34aae76f05626b0795276cf97cc34b8
Jul 24 17:15:47.835277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:47.835238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77" event={"ID":"1a9555d6-bef2-49cd-8a69-ee304a259828","Type":"ContainerStarted","Data":"aadea36161de58d7ab11272861b82e2fe34aae76f05626b0795276cf97cc34b8"}
Jul 24 17:15:47.836390 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:47.836369    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" event={"ID":"8fc5d37e-9881-47f9-ac6d-c723fe4cb208","Type":"ContainerStarted","Data":"b796b111a394b69c2cef46cd1b62886a87697f731075b73642352d7a017302f5"}
Jul 24 17:15:48.503897 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.503861    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"]
Jul 24 17:15:48.508673 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.508657    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.511249 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.511227    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 17:15:48.511368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.511273    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nh49w\""
Jul 24 17:15:48.511368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.511344    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 17:15:48.511368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.511347    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 17:15:48.518860 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.518836    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"]
Jul 24 17:15:48.608184 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.608152    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dc0ead-de9d-4bdc-a91c-772a61d6858b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.608184 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.608186    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.608464 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.608211    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.608464 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.608231    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.608464 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.608257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.608464 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.608295    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7hvf\" (UniqueName: \"kubernetes.io/projected/50dc0ead-de9d-4bdc-a91c-772a61d6858b-kube-api-access-h7hvf\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.608464 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.608374    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.709168 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.709139    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7hvf\" (UniqueName: \"kubernetes.io/projected/50dc0ead-de9d-4bdc-a91c-772a61d6858b-kube-api-access-h7hvf\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.709349 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.709192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.709349 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.709312    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dc0ead-de9d-4bdc-a91c-772a61d6858b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.709461 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.709353    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.709461 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.709390    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.709461 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.709414    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.709461 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.709440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.710496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.710466    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.711766 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.711740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/50dc0ead-de9d-4bdc-a91c-772a61d6858b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.711918 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.711898    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.711971 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.711952    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.712009 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.711954    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/50dc0ead-de9d-4bdc-a91c-772a61d6858b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.716771 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.716750    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7hvf\" (UniqueName: \"kubernetes.io/projected/50dc0ead-de9d-4bdc-a91c-772a61d6858b-kube-api-access-h7hvf\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.717777 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.717756    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50dc0ead-de9d-4bdc-a91c-772a61d6858b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-mkdsh\" (UID: \"50dc0ead-de9d-4bdc-a91c-772a61d6858b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.817874 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.817783    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:48.947542 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:48.947500    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"]
Jul 24 17:15:48.949712 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:15:48.949686    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dc0ead_de9d_4bdc_a91c_772a61d6858b.slice/crio-ffceaaf0c2f5e3f469fe736d71bbdbfb6d2e9c66fb2978c70873dfcaadbede00 WatchSource:0}: Error finding container ffceaaf0c2f5e3f469fe736d71bbdbfb6d2e9c66fb2978c70873dfcaadbede00: Status 404 returned error can't find the container with id ffceaaf0c2f5e3f469fe736d71bbdbfb6d2e9c66fb2978c70873dfcaadbede00
Jul 24 17:15:49.846533 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:49.846459    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh" event={"ID":"50dc0ead-de9d-4bdc-a91c-772a61d6858b","Type":"ContainerStarted","Data":"ffceaaf0c2f5e3f469fe736d71bbdbfb6d2e9c66fb2978c70873dfcaadbede00"}
Jul 24 17:15:51.812318 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:51.812283    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57c7c45669-zx7rw"
Jul 24 17:15:52.694102 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.694058    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:15:52.694200 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.694122    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:15:52.694258 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.694206    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:15:52.694258 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.694145    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:15:52.694258 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.694247    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:15:52.695330 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.695299    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:15:52.695427 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.695396    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:15:52.695483 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.695445    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:15:52.865361 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.865281    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77" event={"ID":"1a9555d6-bef2-49cd-8a69-ee304a259828","Type":"ContainerStarted","Data":"32bcb1f506df2feb64a9863721c6237fca055224aaa3105690a982eac0d66fa7"}
Jul 24 17:15:52.868095 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.867726    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" event={"ID":"8fc5d37e-9881-47f9-ac6d-c723fe4cb208","Type":"ContainerStarted","Data":"e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d"}
Jul 24 17:15:52.872893 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.872734    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh" event={"ID":"50dc0ead-de9d-4bdc-a91c-772a61d6858b","Type":"ContainerStarted","Data":"05d6ca1be900b1de7d9f3d6a71bffcf145cf02d8b74783085b7418a6f13214cf"}
Jul 24 17:15:52.872893 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.872887    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:52.895889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.895827    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77" podStartSLOduration=1.459861714 podStartE2EDuration="6.895804349s" totalImagesPullingTime="5.435942635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:52.893171677 +0000 UTC m=+341.864303853" watchObservedRunningTime="2026-07-24 17:15:52.895804349 +0000 UTC m=+341.866936530"
Jul 24 17:15:52.919512 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.919450    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" podStartSLOduration=1.142694576 podStartE2EDuration="6.919427221s" totalImagesPullingTime="5.776732645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:52.917973056 +0000 UTC m=+341.889105255" watchObservedRunningTime="2026-07-24 17:15:52.919427221 +0000 UTC m=+341.890559404"
Jul 24 17:15:52.941495 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:52.941391    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh" podStartSLOduration=1.199287137 podStartE2EDuration="4.941367957s" totalImagesPullingTime="3.74208082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:15:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:15:52.93994287 +0000 UTC m=+341.911075052" watchObservedRunningTime="2026-07-24 17:15:52.941367957 +0000 UTC m=+341.912500140"
Jul 24 17:15:53.113425 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.113390    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:53.118629 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.118389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:53.777536 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.777496    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:15:53.778936 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.778906    2562 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcngggp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.17:15021/healthz/ready\": dial tcp 10.133.0.17:15021: connect: connection refused" start-of-body=
Jul 24 17:15:53.779091 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.778955    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" podUID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.17:15021/healthz/ready\": dial tcp 10.133.0.17:15021: connect: connection refused"
Jul 24 17:15:53.877219 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.877180    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:53.878302 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.878284    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77"
Jul 24 17:15:53.878982 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.878963    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-mkdsh"
Jul 24 17:15:53.948789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:53.948760    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"]
Jul 24 17:15:54.777230 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:54.777187    2562 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcngggp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.17:15021/healthz/ready\": dial tcp 10.133.0.17:15021: connect: connection refused" start-of-body=
Jul 24 17:15:54.777422 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:54.777256    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" podUID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.17:15021/healthz/ready\": dial tcp 10.133.0.17:15021: connect: connection refused"
Jul 24 17:15:55.777508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:55.777463    2562 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcngggp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.17:15021/healthz/ready\": dial tcp 10.133.0.17:15021: connect: connection refused" start-of-body=
Jul 24 17:15:55.777926 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:55.777525    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" podUID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.17:15021/healthz/ready\": dial tcp 10.133.0.17:15021: connect: connection refused"
Jul 24 17:15:55.885131 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:15:55.885086    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" podUID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" containerName="istio-proxy" containerID="cri-o://e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d" gracePeriod=30
Jul 24 17:16:01.127603 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.127572    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:16:01.222427 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222394    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-envoy\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-token\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222473    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istiod-ca-cert\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222494    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-credential-socket\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222528    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-socket\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222543    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-certs\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222563    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-data\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222603    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfkhg\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-kube-api-access-qfkhg\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222636    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-podinfo\") pod \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\" (UID: \"8fc5d37e-9881-47f9-ac6d-c723fe4cb208\") "
Jul 24 17:16:01.222989 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222965    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:16:01.223087 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222978    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:16:01.223087 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.222978    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-data" (OuterVolumeSpecName: "istio-data") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:16:01.223171 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.223124    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:16:01.223212 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.223181    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:16:01.225079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.225032    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:16:01.225200 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.225086    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-token" (OuterVolumeSpecName: "istio-token") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:16:01.225200 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.225112    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:16:01.225200 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.225161    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-kube-api-access-qfkhg" (OuterVolumeSpecName: "kube-api-access-qfkhg") pod "8fc5d37e-9881-47f9-ac6d-c723fe4cb208" (UID: "8fc5d37e-9881-47f9-ac6d-c723fe4cb208"). InnerVolumeSpecName "kube-api-access-qfkhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:16:01.323719 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323681    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.323719 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323712    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.323719 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323725    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.323967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323736    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.323967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323747    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.323967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323759    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.323967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323770    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.323967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323780    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.323967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.323790    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfkhg\" (UniqueName: \"kubernetes.io/projected/8fc5d37e-9881-47f9-ac6d-c723fe4cb208-kube-api-access-qfkhg\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:16:01.908655 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.908562    2562 generic.go:356] "Generic (PLEG): container finished" podID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" containerID="e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d" exitCode=0
Jul 24 17:16:01.908655 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.908610    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"
Jul 24 17:16:01.908868 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.908653    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" event={"ID":"8fc5d37e-9881-47f9-ac6d-c723fe4cb208","Type":"ContainerDied","Data":"e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d"}
Jul 24 17:16:01.908868 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.908708    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp" event={"ID":"8fc5d37e-9881-47f9-ac6d-c723fe4cb208","Type":"ContainerDied","Data":"b796b111a394b69c2cef46cd1b62886a87697f731075b73642352d7a017302f5"}
Jul 24 17:16:01.908868 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.908732    2562 scope.go:117] "RemoveContainer" containerID="e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d"
Jul 24 17:16:01.916992 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.916974    2562 scope.go:117] "RemoveContainer" containerID="e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d"
Jul 24 17:16:01.917342 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:16:01.917313    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d\": container with ID starting with e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d not found: ID does not exist" containerID="e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d"
Jul 24 17:16:01.917428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.917354    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d"} err="failed to get container status \"e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d\": rpc error: code = NotFound desc = could not find container \"e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d\": container with ID starting with e0e0fc0150528669533e20878a05136c396179cf39ffa80516a4b6cc3e59345d not found: ID does not exist"
Jul 24 17:16:01.926361 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.926341    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"]
Jul 24 17:16:01.932321 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:01.932301    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcngggp"]
Jul 24 17:16:03.507191 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:16:03.507155    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" path="/var/lib/kubelet/pods/8fc5d37e-9881-47f9-ac6d-c723fe4cb208/volumes"
Jul 24 17:18:07.808608 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.808566    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"]
Jul 24 17:18:07.809041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.808859    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" containerName="istio-proxy"
Jul 24 17:18:07.809041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.808869    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" containerName="istio-proxy"
Jul 24 17:18:07.809041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.808918    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fc5d37e-9881-47f9-ac6d-c723fe4cb208" containerName="istio-proxy"
Jul 24 17:18:07.811808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.811787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.814448 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.814422    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-fr29x\""
Jul 24 17:18:07.826579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.826557    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"]
Jul 24 17:18:07.868670 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.868838 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.868838 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868709    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.868838 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868741    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54ab0abd-73ed-4fb7-9250-1345d1437057-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.868838 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868770    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.868838 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868805    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.868838 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868830    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.869080 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868848    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.869080 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.868879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r8rj\" (UniqueName: \"kubernetes.io/projected/54ab0abd-73ed-4fb7-9250-1345d1437057-kube-api-access-8r8rj\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969337 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969302    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969353    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54ab0abd-73ed-4fb7-9250-1345d1437057-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969414    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969450    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969473    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r8rj\" (UniqueName: \"kubernetes.io/projected/54ab0abd-73ed-4fb7-9250-1345d1437057-kube-api-access-8r8rj\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969827    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.969949 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.969921    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.970235 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.970214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.970235 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.970226    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.970393 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.970373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54ab0abd-73ed-4fb7-9250-1345d1437057-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.971787 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.971752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.971906 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.971889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.978071 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.978050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54ab0abd-73ed-4fb7-9250-1345d1437057-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:07.978216 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:07.978197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r8rj\" (UniqueName: \"kubernetes.io/projected/54ab0abd-73ed-4fb7-9250-1345d1437057-kube-api-access-8r8rj\") pod \"maas-default-gateway-openshift-default-78bb78ff78-j6kgc\" (UID: \"54ab0abd-73ed-4fb7-9250-1345d1437057\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:08.121645 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:08.121556    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:08.253514 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:08.253391    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"]
Jul 24 17:18:08.256160 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:18:08.256138    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54ab0abd_73ed_4fb7_9250_1345d1437057.slice/crio-ed8d1ab9f7dffc9ea07d410d7c8f9ed7cdc50c34cf71b0851590d54892b0aeeb WatchSource:0}: Error finding container ed8d1ab9f7dffc9ea07d410d7c8f9ed7cdc50c34cf71b0851590d54892b0aeeb: Status 404 returned error can't find the container with id ed8d1ab9f7dffc9ea07d410d7c8f9ed7cdc50c34cf71b0851590d54892b0aeeb
Jul 24 17:18:08.258362 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:08.258325    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:18:08.258423 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:08.258408    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:18:08.258465 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:08.258450    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:18:08.372341 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:08.372239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" event={"ID":"54ab0abd-73ed-4fb7-9250-1345d1437057","Type":"ContainerStarted","Data":"72c3b11dec6027218ba97f9e4fb3f87d69fbc0be49a0ec306ef58514b27ab8de"}
Jul 24 17:18:08.372341 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:08.372309    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" event={"ID":"54ab0abd-73ed-4fb7-9250-1345d1437057","Type":"ContainerStarted","Data":"ed8d1ab9f7dffc9ea07d410d7c8f9ed7cdc50c34cf71b0851590d54892b0aeeb"}
Jul 24 17:18:08.393850 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:08.393790    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" podStartSLOduration=1.393768578 podStartE2EDuration="1.393768578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:18:08.391707537 +0000 UTC m=+477.362839719" watchObservedRunningTime="2026-07-24 17:18:08.393768578 +0000 UTC m=+477.364900761"
Jul 24 17:18:09.121992 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:09.121955    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:09.126745 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:09.126720    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:09.376172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:09.376093    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:18:09.377154 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:18:09.377138    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:19:04.760596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:04.760501    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-wfcfv"]
Jul 24 17:19:04.763525 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:04.763507    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:04.766218 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:04.766201    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 17:19:04.766351 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:04.766242    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 17:19:04.767441 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:04.767421    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5lnmt\""
Jul 24 17:19:04.774633 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:04.774608    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-wfcfv"]
Jul 24 17:19:04.941555 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:04.941520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28vvh\" (UniqueName: \"kubernetes.io/projected/3e03736f-3a2e-43b1-ac96-8e3f328ff962-kube-api-access-28vvh\") pod \"maas-controller-9c58cc7f8-wfcfv\" (UID: \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\") " pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:04.941728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:04.941571    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e03736f-3a2e-43b1-ac96-8e3f328ff962-webhook-cert\") pod \"maas-controller-9c58cc7f8-wfcfv\" (UID: \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\") " pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:05.042644 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:05.042546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28vvh\" (UniqueName: \"kubernetes.io/projected/3e03736f-3a2e-43b1-ac96-8e3f328ff962-kube-api-access-28vvh\") pod \"maas-controller-9c58cc7f8-wfcfv\" (UID: \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\") " pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:05.042644 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:05.042621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e03736f-3a2e-43b1-ac96-8e3f328ff962-webhook-cert\") pod \"maas-controller-9c58cc7f8-wfcfv\" (UID: \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\") " pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:05.045173 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:05.045146    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e03736f-3a2e-43b1-ac96-8e3f328ff962-webhook-cert\") pod \"maas-controller-9c58cc7f8-wfcfv\" (UID: \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\") " pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:05.051587 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:05.051567    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28vvh\" (UniqueName: \"kubernetes.io/projected/3e03736f-3a2e-43b1-ac96-8e3f328ff962-kube-api-access-28vvh\") pod \"maas-controller-9c58cc7f8-wfcfv\" (UID: \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\") " pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:05.073390 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:05.073363    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:05.203230 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:05.203210    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-wfcfv"]
Jul 24 17:19:05.205724 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:19:05.205692    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e03736f_3a2e_43b1_ac96_8e3f328ff962.slice/crio-77d1f9e347531693ba864ae44db45c89d80fd2306193242bc418516405dc7f61 WatchSource:0}: Error finding container 77d1f9e347531693ba864ae44db45c89d80fd2306193242bc418516405dc7f61: Status 404 returned error can't find the container with id 77d1f9e347531693ba864ae44db45c89d80fd2306193242bc418516405dc7f61
Jul 24 17:19:05.594151 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:05.594112    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv" event={"ID":"3e03736f-3a2e-43b1-ac96-8e3f328ff962","Type":"ContainerStarted","Data":"77d1f9e347531693ba864ae44db45c89d80fd2306193242bc418516405dc7f61"}
Jul 24 17:19:08.609233 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:08.609199    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv" event={"ID":"3e03736f-3a2e-43b1-ac96-8e3f328ff962","Type":"ContainerStarted","Data":"bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504"}
Jul 24 17:19:08.609613 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:08.609316    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:08.628518 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:08.628475    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv" podStartSLOduration=2.288116314 podStartE2EDuration="4.628462841s" totalImagesPullingTime="2.340346527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:19:08.626740631 +0000 UTC m=+537.597872815" watchObservedRunningTime="2026-07-24 17:19:08.628462841 +0000 UTC m=+537.599595022"
Jul 24 17:19:19.615788 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:19.615757    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:19:40.704266 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.704230    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"]
Jul 24 17:19:40.707614 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.707599    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:40.710279 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.710251    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 17:19:40.710407 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.710251    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-k5452\""
Jul 24 17:19:40.719688 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.719664    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"]
Jul 24 17:19:40.812172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.812139    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rznf\" (UniqueName: \"kubernetes.io/projected/eccc9437-3a8e-461e-9404-1b7069db6edf-kube-api-access-5rznf\") pod \"payload-pre-processing-687dbb6cf6-vzv9n\" (UID: \"eccc9437-3a8e-461e-9404-1b7069db6edf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:40.812172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.812176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eccc9437-3a8e-461e-9404-1b7069db6edf-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vzv9n\" (UID: \"eccc9437-3a8e-461e-9404-1b7069db6edf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:40.913055 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.912981    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rznf\" (UniqueName: \"kubernetes.io/projected/eccc9437-3a8e-461e-9404-1b7069db6edf-kube-api-access-5rznf\") pod \"payload-pre-processing-687dbb6cf6-vzv9n\" (UID: \"eccc9437-3a8e-461e-9404-1b7069db6edf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:40.913258 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.913070    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eccc9437-3a8e-461e-9404-1b7069db6edf-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vzv9n\" (UID: \"eccc9437-3a8e-461e-9404-1b7069db6edf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:40.913789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.913769    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eccc9437-3a8e-461e-9404-1b7069db6edf-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vzv9n\" (UID: \"eccc9437-3a8e-461e-9404-1b7069db6edf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:40.922526 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:40.922492    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rznf\" (UniqueName: \"kubernetes.io/projected/eccc9437-3a8e-461e-9404-1b7069db6edf-kube-api-access-5rznf\") pod \"payload-pre-processing-687dbb6cf6-vzv9n\" (UID: \"eccc9437-3a8e-461e-9404-1b7069db6edf\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:41.017698 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:41.017598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:41.155541 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:41.155513    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"]
Jul 24 17:19:41.158326 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:19:41.158288    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccc9437_3a8e_461e_9404_1b7069db6edf.slice/crio-07c58ebdeda429b02eb0f11387cd466e188101b2a963bca520cecaae78db46cf WatchSource:0}: Error finding container 07c58ebdeda429b02eb0f11387cd466e188101b2a963bca520cecaae78db46cf: Status 404 returned error can't find the container with id 07c58ebdeda429b02eb0f11387cd466e188101b2a963bca520cecaae78db46cf
Jul 24 17:19:41.742336 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:41.742300    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n" event={"ID":"eccc9437-3a8e-461e-9404-1b7069db6edf","Type":"ContainerStarted","Data":"07c58ebdeda429b02eb0f11387cd466e188101b2a963bca520cecaae78db46cf"}
Jul 24 17:19:44.784381 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:44.784342    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n" event={"ID":"eccc9437-3a8e-461e-9404-1b7069db6edf","Type":"ContainerStarted","Data":"7d192dca86a93018dbf6b536b9c26317f20a7971ae8fe373a928987941f23a83"}
Jul 24 17:19:44.784788 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:44.784451    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:19:44.814271 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:44.814214    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n" podStartSLOduration=1.967328307 podStartE2EDuration="4.814200358s" totalImagesPullingTime="2.846872051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:19:44.814075286 +0000 UTC m=+573.785207468" watchObservedRunningTime="2026-07-24 17:19:44.814200358 +0000 UTC m=+573.785332567"
Jul 24 17:19:48.802933 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:48.802903    2562 generic.go:356] "Generic (PLEG): container finished" podID="54ab0abd-73ed-4fb7-9250-1345d1437057" containerID="72c3b11dec6027218ba97f9e4fb3f87d69fbc0be49a0ec306ef58514b27ab8de" exitCode=0
Jul 24 17:19:48.803378 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:48.802986    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" event={"ID":"54ab0abd-73ed-4fb7-9250-1345d1437057","Type":"ContainerDied","Data":"72c3b11dec6027218ba97f9e4fb3f87d69fbc0be49a0ec306ef58514b27ab8de"}
Jul 24 17:19:48.803378 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:48.803368    2562 scope.go:117] "RemoveContainer" containerID="72c3b11dec6027218ba97f9e4fb3f87d69fbc0be49a0ec306ef58514b27ab8de"
Jul 24 17:19:48.804289 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:48.804256    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:19:48.804400 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:48.804327    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:19:48.804400 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:48.804359    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:19:49.809771 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:49.809733    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" event={"ID":"54ab0abd-73ed-4fb7-9250-1345d1437057","Type":"ContainerStarted","Data":"07420cfbe8b5883c84d6ae09cce761ec9358dd1ef4c4287323dd8f4e9a4d7d4a"}
Jul 24 17:19:50.122420 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:50.122318    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:19:50.123476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:50.123446    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j6kgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 24 17:19:50.123652 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:50.123523    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" podUID="54ab0abd-73ed-4fb7-9250-1345d1437057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 24 17:19:51.122897 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:51.122868    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j6kgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 24 17:19:51.123265 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:51.122934    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" podUID="54ab0abd-73ed-4fb7-9250-1345d1437057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 24 17:19:52.122892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:52.122854    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j6kgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 24 17:19:52.123107 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:52.122913    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" podUID="54ab0abd-73ed-4fb7-9250-1345d1437057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 24 17:19:53.122194 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:53.122155    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-j6kgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused" start-of-body=
Jul 24 17:19:53.122410 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:53.122231    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc" podUID="54ab0abd-73ed-4fb7-9250-1345d1437057" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.20:15021/healthz/ready\": dial tcp 10.133.0.20:15021: connect: connection refused"
Jul 24 17:19:54.126100 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:54.126066    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:19:54.126488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:54.126341    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:19:54.133054 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:54.133011    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-j6kgc"
Jul 24 17:19:55.790205 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:19:55.790172    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vzv9n"
Jul 24 17:20:11.416819 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:11.416788    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:20:11.421805 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:11.421771    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:20:47.864809 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.864728    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"]
Jul 24 17:20:47.866941 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.866922    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:47.869625 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.869608    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 17:20:47.869744 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.869725    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 24 17:20:47.870922 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.870900    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 17:20:47.870922 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.870931    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7nbv9\""
Jul 24 17:20:47.878496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.878472    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"]
Jul 24 17:20:47.952619 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.952583    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:47.952790 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.952661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f27589c9-42d0-4436-8c96-9c8d23ae389b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:47.952790 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.952710    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5cr7\" (UniqueName: \"kubernetes.io/projected/f27589c9-42d0-4436-8c96-9c8d23ae389b-kube-api-access-z5cr7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:47.952790 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.952751    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:47.952790 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.952779    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:47.952931 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:47.952800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053319 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053283    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053561 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053335    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f27589c9-42d0-4436-8c96-9c8d23ae389b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053561 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053524    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5cr7\" (UniqueName: \"kubernetes.io/projected/f27589c9-42d0-4436-8c96-9c8d23ae389b-kube-api-access-z5cr7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053561 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053737 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053573    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053737 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053737 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053914 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053894    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.053970 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.053956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.055932 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.055907    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f27589c9-42d0-4436-8c96-9c8d23ae389b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.056147 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.056130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f27589c9-42d0-4436-8c96-9c8d23ae389b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.061976 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.061957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5cr7\" (UniqueName: \"kubernetes.io/projected/f27589c9-42d0-4436-8c96-9c8d23ae389b-kube-api-access-z5cr7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9x9nb\" (UID: \"f27589c9-42d0-4436-8c96-9c8d23ae389b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.176747 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.176670    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:48.310081 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.308127    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"]
Jul 24 17:20:48.313333 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:48.313312    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:20:49.045817 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:49.045771    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb" event={"ID":"f27589c9-42d0-4436-8c96-9c8d23ae389b","Type":"ContainerStarted","Data":"82a7309cbdfa9f9b44a1d7dbc18b8532ca2aa9769302ba3d55027080199c6039"}
Jul 24 17:20:50.051592 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:50.051550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb" event={"ID":"f27589c9-42d0-4436-8c96-9c8d23ae389b","Type":"ContainerStarted","Data":"76ce44706bf9c752fab2a706d01a675c3786e13ab2814e5b46c5b088e8d95ff0"}
Jul 24 17:20:50.072130 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:50.072079    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb" podStartSLOduration=2.090726703 podStartE2EDuration="3.072062931s" totalImagesPullingTime="981.336228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:20:50.069886025 +0000 UTC m=+639.041018206" watchObservedRunningTime="2026-07-24 17:20:50.072062931 +0000 UTC m=+639.043195112"
Jul 24 17:20:58.177709 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:58.177681    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:58.178160 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:58.177944    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:58.190713 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:58.190687    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:20:59.100470 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:20:59.100441    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9x9nb"
Jul 24 17:22:54.291118 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.291083    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-wfcfv"]
Jul 24 17:22:54.291740 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.291298    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv" podUID="3e03736f-3a2e-43b1-ac96-8e3f328ff962" containerName="manager" containerID="cri-o://bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504" gracePeriod=30
Jul 24 17:22:54.536424 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.536398    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:22:54.565522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.565442    2562 generic.go:356] "Generic (PLEG): container finished" podID="3e03736f-3a2e-43b1-ac96-8e3f328ff962" containerID="bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504" exitCode=0
Jul 24 17:22:54.565522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.565494    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv"
Jul 24 17:22:54.565728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.565530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv" event={"ID":"3e03736f-3a2e-43b1-ac96-8e3f328ff962","Type":"ContainerDied","Data":"bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504"}
Jul 24 17:22:54.565728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.565594    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-wfcfv" event={"ID":"3e03736f-3a2e-43b1-ac96-8e3f328ff962","Type":"ContainerDied","Data":"77d1f9e347531693ba864ae44db45c89d80fd2306193242bc418516405dc7f61"}
Jul 24 17:22:54.565728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.565627    2562 scope.go:117] "RemoveContainer" containerID="bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504"
Jul 24 17:22:54.574145 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.574125    2562 scope.go:117] "RemoveContainer" containerID="bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504"
Jul 24 17:22:54.574424 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:22:54.574394    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504\": container with ID starting with bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504 not found: ID does not exist" containerID="bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504"
Jul 24 17:22:54.574498 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.574423    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504"} err="failed to get container status \"bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504\": rpc error: code = NotFound desc = could not find container \"bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504\": container with ID starting with bcdfc8205b951eb0b690df2ac3f9d0155d3cafeb3f8baf419d94213cde2c3504 not found: ID does not exist"
Jul 24 17:22:54.631463 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.631433    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28vvh\" (UniqueName: \"kubernetes.io/projected/3e03736f-3a2e-43b1-ac96-8e3f328ff962-kube-api-access-28vvh\") pod \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\" (UID: \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\") "
Jul 24 17:22:54.631659 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.631471    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e03736f-3a2e-43b1-ac96-8e3f328ff962-webhook-cert\") pod \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\" (UID: \"3e03736f-3a2e-43b1-ac96-8e3f328ff962\") "
Jul 24 17:22:54.633789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.633753    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e03736f-3a2e-43b1-ac96-8e3f328ff962-kube-api-access-28vvh" (OuterVolumeSpecName: "kube-api-access-28vvh") pod "3e03736f-3a2e-43b1-ac96-8e3f328ff962" (UID: "3e03736f-3a2e-43b1-ac96-8e3f328ff962"). InnerVolumeSpecName "kube-api-access-28vvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:22:54.633789 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.633785    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e03736f-3a2e-43b1-ac96-8e3f328ff962-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3e03736f-3a2e-43b1-ac96-8e3f328ff962" (UID: "3e03736f-3a2e-43b1-ac96-8e3f328ff962"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:22:54.732395 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.732350    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28vvh\" (UniqueName: \"kubernetes.io/projected/3e03736f-3a2e-43b1-ac96-8e3f328ff962-kube-api-access-28vvh\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:22:54.732395 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.732386    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3e03736f-3a2e-43b1-ac96-8e3f328ff962-webhook-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:22:54.885448 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.885393    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-wfcfv"]
Jul 24 17:22:54.887117 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:54.887094    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-wfcfv"]
Jul 24 17:22:55.504995 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:22:55.504961    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e03736f-3a2e-43b1-ac96-8e3f328ff962" path="/var/lib/kubelet/pods/3e03736f-3a2e-43b1-ac96-8e3f328ff962/volumes"
Jul 24 17:23:55.722388 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.722350    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-lr9qz"]
Jul 24 17:23:55.722803 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.722619    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e03736f-3a2e-43b1-ac96-8e3f328ff962" containerName="manager"
Jul 24 17:23:55.722803 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.722628    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e03736f-3a2e-43b1-ac96-8e3f328ff962" containerName="manager"
Jul 24 17:23:55.722803 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.722692    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e03736f-3a2e-43b1-ac96-8e3f328ff962" containerName="manager"
Jul 24 17:23:55.724447 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.724431    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:55.726991 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.726968    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 17:23:55.728241 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.728219    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5lnmt\""
Jul 24 17:23:55.728335 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.728315    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 17:23:55.735397 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.735374    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-lr9qz"]
Jul 24 17:23:55.827580 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.827553    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-webhook-cert\") pod \"maas-controller-9c58cc7f8-lr9qz\" (UID: \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\") " pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:55.827760 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.827593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dlxm\" (UniqueName: \"kubernetes.io/projected/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-kube-api-access-6dlxm\") pod \"maas-controller-9c58cc7f8-lr9qz\" (UID: \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\") " pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:55.928216 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.928182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-webhook-cert\") pod \"maas-controller-9c58cc7f8-lr9qz\" (UID: \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\") " pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:55.928341 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.928231    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dlxm\" (UniqueName: \"kubernetes.io/projected/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-kube-api-access-6dlxm\") pod \"maas-controller-9c58cc7f8-lr9qz\" (UID: \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\") " pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:55.930858 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.930827    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-webhook-cert\") pod \"maas-controller-9c58cc7f8-lr9qz\" (UID: \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\") " pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:55.936182 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:55.936150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dlxm\" (UniqueName: \"kubernetes.io/projected/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-kube-api-access-6dlxm\") pod \"maas-controller-9c58cc7f8-lr9qz\" (UID: \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\") " pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:56.034668 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:56.034576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:56.166428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:56.166404    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-lr9qz"]
Jul 24 17:23:56.169054 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:23:56.169012    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b14eeb1_a85b_4d5f_bcf4_f26afcf9b42c.slice/crio-a94a3891f888c67384ca809b5ea8727a19acaac19d66751137c337808f75909d WatchSource:0}: Error finding container a94a3891f888c67384ca809b5ea8727a19acaac19d66751137c337808f75909d: Status 404 returned error can't find the container with id a94a3891f888c67384ca809b5ea8727a19acaac19d66751137c337808f75909d
Jul 24 17:23:56.814590 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:56.814503    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz" event={"ID":"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c","Type":"ContainerStarted","Data":"97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17"}
Jul 24 17:23:56.814590 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:56.814559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz" event={"ID":"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c","Type":"ContainerStarted","Data":"a94a3891f888c67384ca809b5ea8727a19acaac19d66751137c337808f75909d"}
Jul 24 17:23:56.814990 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:56.814638    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:23:56.831550 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:23:56.831500    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz" podStartSLOduration=1.463799927 podStartE2EDuration="1.831486371s" totalImagesPullingTime="367.686444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:23:56.830225413 +0000 UTC m=+825.801357609" watchObservedRunningTime="2026-07-24 17:23:56.831486371 +0000 UTC m=+825.802618552"
Jul 24 17:24:07.823491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:07.823461    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:24:12.452895 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.452857    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-lr9qz"]
Jul 24 17:24:12.453455 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.453081    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz" podUID="9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c" containerName="manager" containerID="cri-o://97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17" gracePeriod=30
Jul 24 17:24:12.695593 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.695565    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:24:12.774527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.774446    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dlxm\" (UniqueName: \"kubernetes.io/projected/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-kube-api-access-6dlxm\") pod \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\" (UID: \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\") "
Jul 24 17:24:12.774527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.774516    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-webhook-cert\") pod \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\" (UID: \"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c\") "
Jul 24 17:24:12.776618 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.776580    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c" (UID: "9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:24:12.776618 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.776580    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-kube-api-access-6dlxm" (OuterVolumeSpecName: "kube-api-access-6dlxm") pod "9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c" (UID: "9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c"). InnerVolumeSpecName "kube-api-access-6dlxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:24:12.875214 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.875174    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dlxm\" (UniqueName: \"kubernetes.io/projected/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-kube-api-access-6dlxm\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:24:12.875214 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.875206    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c-webhook-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:24:12.883723 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.883695    2562 generic.go:356] "Generic (PLEG): container finished" podID="9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c" containerID="97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17" exitCode=0
Jul 24 17:24:12.883877 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.883740    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz"
Jul 24 17:24:12.883877 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.883758    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz" event={"ID":"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c","Type":"ContainerDied","Data":"97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17"}
Jul 24 17:24:12.883877 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.883800    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-lr9qz" event={"ID":"9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c","Type":"ContainerDied","Data":"a94a3891f888c67384ca809b5ea8727a19acaac19d66751137c337808f75909d"}
Jul 24 17:24:12.883877 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.883821    2562 scope.go:117] "RemoveContainer" containerID="97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17"
Jul 24 17:24:12.892443 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.892421    2562 scope.go:117] "RemoveContainer" containerID="97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17"
Jul 24 17:24:12.892703 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:24:12.892685    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17\": container with ID starting with 97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17 not found: ID does not exist" containerID="97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17"
Jul 24 17:24:12.892770 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.892712    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17"} err="failed to get container status \"97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17\": rpc error: code = NotFound desc = could not find container \"97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17\": container with ID starting with 97657c2ff37c5844b78c782b4db53bef12165a41931b7a44745f6f6da995db17 not found: ID does not exist"
Jul 24 17:24:12.905721 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.905695    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-lr9qz"]
Jul 24 17:24:12.911247 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:12.911222    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-lr9qz"]
Jul 24 17:24:13.504555 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.504515    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c" path="/var/lib/kubelet/pods/9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c/volumes"
Jul 24 17:24:13.690400 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.690367    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-zkhg9"]
Jul 24 17:24:13.690658 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.690646    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c" containerName="manager"
Jul 24 17:24:13.690697 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.690659    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c" containerName="manager"
Jul 24 17:24:13.690735 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.690716    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b14eeb1-a85b-4d5f-bcf4-f26afcf9b42c" containerName="manager"
Jul 24 17:24:13.694956 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.694939    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:13.699110 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.699088    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 17:24:13.699110 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.699102    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 17:24:13.699110 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.699091    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5lnmt\""
Jul 24 17:24:13.709662 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.709605    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-zkhg9"]
Jul 24 17:24:13.782073 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.781953    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkhp7\" (UniqueName: \"kubernetes.io/projected/358bf28d-78cf-4495-8a8d-b3086513b70c-kube-api-access-qkhp7\") pod \"maas-controller-9c58cc7f8-zkhg9\" (UID: \"358bf28d-78cf-4495-8a8d-b3086513b70c\") " pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:13.782073 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.782010    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/358bf28d-78cf-4495-8a8d-b3086513b70c-webhook-cert\") pod \"maas-controller-9c58cc7f8-zkhg9\" (UID: \"358bf28d-78cf-4495-8a8d-b3086513b70c\") " pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:13.883153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.883118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkhp7\" (UniqueName: \"kubernetes.io/projected/358bf28d-78cf-4495-8a8d-b3086513b70c-kube-api-access-qkhp7\") pod \"maas-controller-9c58cc7f8-zkhg9\" (UID: \"358bf28d-78cf-4495-8a8d-b3086513b70c\") " pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:13.883257 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.883185    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/358bf28d-78cf-4495-8a8d-b3086513b70c-webhook-cert\") pod \"maas-controller-9c58cc7f8-zkhg9\" (UID: \"358bf28d-78cf-4495-8a8d-b3086513b70c\") " pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:13.886112 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.886088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/358bf28d-78cf-4495-8a8d-b3086513b70c-webhook-cert\") pod \"maas-controller-9c58cc7f8-zkhg9\" (UID: \"358bf28d-78cf-4495-8a8d-b3086513b70c\") " pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:13.892692 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:13.892660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkhp7\" (UniqueName: \"kubernetes.io/projected/358bf28d-78cf-4495-8a8d-b3086513b70c-kube-api-access-qkhp7\") pod \"maas-controller-9c58cc7f8-zkhg9\" (UID: \"358bf28d-78cf-4495-8a8d-b3086513b70c\") " pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:14.004366 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:14.004326    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:14.137913 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:14.137892    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c58cc7f8-zkhg9"]
Jul 24 17:24:14.140124 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:24:14.140097    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358bf28d_78cf_4495_8a8d_b3086513b70c.slice/crio-d3dcbed3fb8355334ca2203f7befe32147b359bac0476d697485935fdb154165 WatchSource:0}: Error finding container d3dcbed3fb8355334ca2203f7befe32147b359bac0476d697485935fdb154165: Status 404 returned error can't find the container with id d3dcbed3fb8355334ca2203f7befe32147b359bac0476d697485935fdb154165
Jul 24 17:24:14.894246 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:14.894160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-zkhg9" event={"ID":"358bf28d-78cf-4495-8a8d-b3086513b70c","Type":"ContainerStarted","Data":"b928cbd1c00a4e41b915238e8aaa0d5ee5192f3d57fcb42683d9f01194dc3995"}
Jul 24 17:24:14.894246 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:14.894215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c58cc7f8-zkhg9" event={"ID":"358bf28d-78cf-4495-8a8d-b3086513b70c","Type":"ContainerStarted","Data":"d3dcbed3fb8355334ca2203f7befe32147b359bac0476d697485935fdb154165"}
Jul 24 17:24:14.894649 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:14.894289    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:24:14.912910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:14.912856    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-9c58cc7f8-zkhg9" podStartSLOduration=1.572669154 podStartE2EDuration="1.912840063s" totalImagesPullingTime="340.170909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:24:14.910259348 +0000 UTC m=+843.881391530" watchObservedRunningTime="2026-07-24 17:24:14.912840063 +0000 UTC m=+843.883972243"
Jul 24 17:24:25.901328 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:24:25.901294    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9c58cc7f8-zkhg9"
Jul 24 17:25:11.444279 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:25:11.444195    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:25:11.446302 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:25:11.446279    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:30:00.137454 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:00.137419    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7"]
Jul 24 17:30:00.140833 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:00.140811    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7"
Jul 24 17:30:00.143639 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:00.143622    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 17:30:00.149305 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:00.149262    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7"]
Jul 24 17:30:00.152779 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:00.152757    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7"
Jul 24 17:30:00.282844 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:00.282819    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7"]
Jul 24 17:30:00.285124 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:30:00.285095    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6762fda9_de50_4a3f_80b3_57b92c778208.slice/crio-834023d2e7b77ab9a4271a793890b8968824bcbdac2992a026fa6fe60ec85347 WatchSource:0}: Error finding container 834023d2e7b77ab9a4271a793890b8968824bcbdac2992a026fa6fe60ec85347: Status 404 returned error can't find the container with id 834023d2e7b77ab9a4271a793890b8968824bcbdac2992a026fa6fe60ec85347
Jul 24 17:30:00.286843 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:00.286828    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:30:01.295430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:01.295392    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7" event={"ID":"6762fda9-de50-4a3f-80b3-57b92c778208","Type":"ContainerStarted","Data":"834023d2e7b77ab9a4271a793890b8968824bcbdac2992a026fa6fe60ec85347"}
Jul 24 17:30:03.306421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:03.306342    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7" event={"ID":"6762fda9-de50-4a3f-80b3-57b92c778208","Type":"ContainerStarted","Data":"d6e50d35714a5ee6635397061972b3e758b3df9e373b1eabe7f2946c118fa2f4"}
Jul 24 17:30:03.322105 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:03.322059    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7" podStartSLOduration=0.566169517 podStartE2EDuration="3.322046707s" totalImagesPullingTime="2.75587719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:30:03.32161954 +0000 UTC m=+1192.292751722" watchObservedRunningTime="2026-07-24 17:30:03.322046707 +0000 UTC m=+1192.293178885"
Jul 24 17:30:04.312287 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:04.312255    2562 generic.go:356] "Generic (PLEG): container finished" podID="6762fda9-de50-4a3f-80b3-57b92c778208" containerID="d6e50d35714a5ee6635397061972b3e758b3df9e373b1eabe7f2946c118fa2f4" exitCode=0
Jul 24 17:30:04.312665 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:04.312345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7" event={"ID":"6762fda9-de50-4a3f-80b3-57b92c778208","Type":"ContainerDied","Data":"d6e50d35714a5ee6635397061972b3e758b3df9e373b1eabe7f2946c118fa2f4"}
Jul 24 17:30:05.442705 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:05.442677    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7"
Jul 24 17:30:06.321295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:06.321258    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7" event={"ID":"6762fda9-de50-4a3f-80b3-57b92c778208","Type":"ContainerDied","Data":"834023d2e7b77ab9a4271a793890b8968824bcbdac2992a026fa6fe60ec85347"}
Jul 24 17:30:06.321475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:06.321307    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="834023d2e7b77ab9a4271a793890b8968824bcbdac2992a026fa6fe60ec85347"
Jul 24 17:30:06.321475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:06.321271    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748570-rtqt7"
Jul 24 17:30:11.469199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:11.469174    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:30:11.473407 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:30:11.473387    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:32:36.366842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.366757    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"]
Jul 24 17:32:36.367387 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.367207    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6762fda9-de50-4a3f-80b3-57b92c778208" containerName="cleanup"
Jul 24 17:32:36.367387 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.367222    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6762fda9-de50-4a3f-80b3-57b92c778208" containerName="cleanup"
Jul 24 17:32:36.367387 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.367285    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6762fda9-de50-4a3f-80b3-57b92c778208" containerName="cleanup"
Jul 24 17:32:36.370032 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.369999    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.372868 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.372841    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-36c07d19-kserve-self-signed-certs\""
Jul 24 17:32:36.384888 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.384864    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"]
Jul 24 17:32:36.467989 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.467950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-model-cache\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.468192 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.468002    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-tmp-dir\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.468192 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.468081    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-dshm\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.468192 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.468142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-452xq\" (UniqueName: \"kubernetes.io/projected/dfb49810-ee82-4615-9464-7e69dcfa2416-kube-api-access-452xq\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.468349 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.468243    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb49810-ee82-4615-9464-7e69dcfa2416-tls-certs\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.468349 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.468280    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-home\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569386 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-home\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569386 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-model-cache\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569625 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569544    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-tmp-dir\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569625 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-dshm\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569669    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-452xq\" (UniqueName: \"kubernetes.io/projected/dfb49810-ee82-4615-9464-7e69dcfa2416-kube-api-access-452xq\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569781 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569746    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb49810-ee82-4615-9464-7e69dcfa2416-tls-certs\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569831 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-home\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569880 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-model-cache\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.569880 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.569868    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-tmp-dir\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.572113 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.572088    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-dshm\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.572358 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.572340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb49810-ee82-4615-9464-7e69dcfa2416-tls-certs\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.579187 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.579161    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-452xq\" (UniqueName: \"kubernetes.io/projected/dfb49810-ee82-4615-9464-7e69dcfa2416-kube-api-access-452xq\") pod \"e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") " pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.679310 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.679225    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:36.813215 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.813189    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"]
Jul 24 17:32:36.815874 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:32:36.815836    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfb49810_ee82_4615_9464_7e69dcfa2416.slice/crio-38f4462302f1d219ce8a3e78e93c574fd5bcab3006c0708b224bb49475c9829a WatchSource:0}: Error finding container 38f4462302f1d219ce8a3e78e93c574fd5bcab3006c0708b224bb49475c9829a: Status 404 returned error can't find the container with id 38f4462302f1d219ce8a3e78e93c574fd5bcab3006c0708b224bb49475c9829a
Jul 24 17:32:36.959505 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.959473    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt" event={"ID":"dfb49810-ee82-4615-9464-7e69dcfa2416","Type":"ContainerStarted","Data":"7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6"}
Jul 24 17:32:36.959708 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.959517    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt" event={"ID":"dfb49810-ee82-4615-9464-7e69dcfa2416","Type":"ContainerStarted","Data":"38f4462302f1d219ce8a3e78e93c574fd5bcab3006c0708b224bb49475c9829a"}
Jul 24 17:32:36.976966 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:36.976920    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt" podStartSLOduration=0.976903263 podStartE2EDuration="976.903263ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:32:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:32:36.976786244 +0000 UTC m=+1345.947918425" watchObservedRunningTime="2026-07-24 17:32:36.976903263 +0000 UTC m=+1345.948035487"
Jul 24 17:32:46.680191 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:46.680099    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:46.680191 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:46.680154    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:46.692769 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:46.692738    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:32:47.013002 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:32:47.012969    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:33:24.281767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:24.281734    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"]
Jul 24 17:33:39.300860 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.300796    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt" podUID="dfb49810-ee82-4615-9464-7e69dcfa2416" containerName="main" containerID="cri-o://7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6" gracePeriod=45
Jul 24 17:33:39.545219 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.545195    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:33:39.681936 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.681858    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-dshm\") pod \"dfb49810-ee82-4615-9464-7e69dcfa2416\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") "
Jul 24 17:33:39.681936 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.681935    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-model-cache\") pod \"dfb49810-ee82-4615-9464-7e69dcfa2416\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") "
Jul 24 17:33:39.682159 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.681969    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb49810-ee82-4615-9464-7e69dcfa2416-tls-certs\") pod \"dfb49810-ee82-4615-9464-7e69dcfa2416\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") "
Jul 24 17:33:39.682159 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.681987    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-tmp-dir\") pod \"dfb49810-ee82-4615-9464-7e69dcfa2416\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") "
Jul 24 17:33:39.682159 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.682004    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-home\") pod \"dfb49810-ee82-4615-9464-7e69dcfa2416\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") "
Jul 24 17:33:39.682159 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.682044    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-452xq\" (UniqueName: \"kubernetes.io/projected/dfb49810-ee82-4615-9464-7e69dcfa2416-kube-api-access-452xq\") pod \"dfb49810-ee82-4615-9464-7e69dcfa2416\" (UID: \"dfb49810-ee82-4615-9464-7e69dcfa2416\") "
Jul 24 17:33:39.682357 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.682227    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-model-cache" (OuterVolumeSpecName: "model-cache") pod "dfb49810-ee82-4615-9464-7e69dcfa2416" (UID: "dfb49810-ee82-4615-9464-7e69dcfa2416"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:39.682422 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.682353    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-home" (OuterVolumeSpecName: "home") pod "dfb49810-ee82-4615-9464-7e69dcfa2416" (UID: "dfb49810-ee82-4615-9464-7e69dcfa2416"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:39.682422 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.682372    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dfb49810-ee82-4615-9464-7e69dcfa2416" (UID: "dfb49810-ee82-4615-9464-7e69dcfa2416"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:39.684265 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.684242    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-dshm" (OuterVolumeSpecName: "dshm") pod "dfb49810-ee82-4615-9464-7e69dcfa2416" (UID: "dfb49810-ee82-4615-9464-7e69dcfa2416"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:33:39.684385 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.684362    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfb49810-ee82-4615-9464-7e69dcfa2416-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dfb49810-ee82-4615-9464-7e69dcfa2416" (UID: "dfb49810-ee82-4615-9464-7e69dcfa2416"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:33:39.684436 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.684415    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfb49810-ee82-4615-9464-7e69dcfa2416-kube-api-access-452xq" (OuterVolumeSpecName: "kube-api-access-452xq") pod "dfb49810-ee82-4615-9464-7e69dcfa2416" (UID: "dfb49810-ee82-4615-9464-7e69dcfa2416"). InnerVolumeSpecName "kube-api-access-452xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:33:39.783422 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.783391    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-model-cache\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:33:39.783422 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.783416    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfb49810-ee82-4615-9464-7e69dcfa2416-tls-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:33:39.783596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.783444    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-tmp-dir\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:33:39.783596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.783452    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-home\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:33:39.783596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.783461    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-452xq\" (UniqueName: \"kubernetes.io/projected/dfb49810-ee82-4615-9464-7e69dcfa2416-kube-api-access-452xq\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:33:39.783596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:39.783469    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfb49810-ee82-4615-9464-7e69dcfa2416-dshm\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:33:40.219712 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.219674    2562 generic.go:356] "Generic (PLEG): container finished" podID="dfb49810-ee82-4615-9464-7e69dcfa2416" containerID="7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6" exitCode=0
Jul 24 17:33:40.219902 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.219718    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"
Jul 24 17:33:40.219902 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.219762    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt" event={"ID":"dfb49810-ee82-4615-9464-7e69dcfa2416","Type":"ContainerDied","Data":"7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6"}
Jul 24 17:33:40.219902 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.219812    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt" event={"ID":"dfb49810-ee82-4615-9464-7e69dcfa2416","Type":"ContainerDied","Data":"38f4462302f1d219ce8a3e78e93c574fd5bcab3006c0708b224bb49475c9829a"}
Jul 24 17:33:40.219902 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.219842    2562 scope.go:117] "RemoveContainer" containerID="7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6"
Jul 24 17:33:40.228332 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.228317    2562 scope.go:117] "RemoveContainer" containerID="7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6"
Jul 24 17:33:40.228607 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:33:40.228589    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6\": container with ID starting with 7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6 not found: ID does not exist" containerID="7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6"
Jul 24 17:33:40.228650 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.228616    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6"} err="failed to get container status \"7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6\": rpc error: code = NotFound desc = could not find container \"7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6\": container with ID starting with 7394be74417ab7eea76a1e4162f1ff23b2413aebdcdb9c18c8b30269983942e6 not found: ID does not exist"
Jul 24 17:33:40.243297 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.243273    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"]
Jul 24 17:33:40.249149 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:40.249123    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-36c07d19-kserve-7b4f8fb94c-794dt"]
Jul 24 17:33:41.505061 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:33:41.505013    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfb49810-ee82-4615-9464-7e69dcfa2416" path="/var/lib/kubelet/pods/dfb49810-ee82-4615-9464-7e69dcfa2416/volumes"
Jul 24 17:35:11.499893 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:35:11.499862    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:35:11.504392 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:35:11.504370    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:36:59.658615 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.658580    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"]
Jul 24 17:36:59.659163 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.658873    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfb49810-ee82-4615-9464-7e69dcfa2416" containerName="main"
Jul 24 17:36:59.659163 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.658883    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfb49810-ee82-4615-9464-7e69dcfa2416" containerName="main"
Jul 24 17:36:59.659163 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.658944    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfb49810-ee82-4615-9464-7e69dcfa2416" containerName="main"
Jul 24 17:36:59.661741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.661722    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.664474 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.664453    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8630be2b-openshift-default-dockercfg-ksztm\""
Jul 24 17:36:59.684262 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.684143    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"]
Jul 24 17:36:59.717944 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.717900    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-credential-socket\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.717944 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.717941    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78289f81-c7a1-46f2-b070-65d05acd3fd4-istiod-ca-cert\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.718199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.717959    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-certs\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.718199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.718053    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-data\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.718199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.718098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9s75\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-kube-api-access-t9s75\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.718199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.718116    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-envoy\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.718199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.718141    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-token\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.718199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.718168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-podinfo\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.718408 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.718219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-socket\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819378    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-credential-socket\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78289f81-c7a1-46f2-b070-65d05acd3fd4-istiod-ca-cert\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-certs\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-data\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9s75\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-kube-api-access-t9s75\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-envoy\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-token\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819558    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-podinfo\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.819739 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-socket\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.820059 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819942    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-certs\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.820059 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.819998    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-data\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.820151 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.820103    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-socket\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.820151 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.820143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-credential-socket\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.820256 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.820236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78289f81-c7a1-46f2-b070-65d05acd3fd4-istiod-ca-cert\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.822246 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.822225    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-envoy\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.822437 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.822422    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-podinfo\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.827780 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.827750    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9s75\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-kube-api-access-t9s75\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.828263 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.828243    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-token\") pod \"e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") " pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:36:59.971363 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:36:59.971317    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:00.109743 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:00.109718    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"]
Jul 24 17:37:00.112413 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:37:00.112384    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78289f81_c7a1_46f2_b070_65d05acd3fd4.slice/crio-38b9a65a9a902bd6745234915a426df299e0b9b71ccf962683c85d0d05f87e1f WatchSource:0}: Error finding container 38b9a65a9a902bd6745234915a426df299e0b9b71ccf962683c85d0d05f87e1f: Status 404 returned error can't find the container with id 38b9a65a9a902bd6745234915a426df299e0b9b71ccf962683c85d0d05f87e1f
Jul 24 17:37:00.114657 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:00.114616    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:00.114752 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:00.114691    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:00.114752 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:00.114721    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:01.031448 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:01.031407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" event={"ID":"78289f81-c7a1-46f2-b070-65d05acd3fd4","Type":"ContainerStarted","Data":"4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab"}
Jul 24 17:37:01.031853 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:01.031470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" event={"ID":"78289f81-c7a1-46f2-b070-65d05acd3fd4","Type":"ContainerStarted","Data":"38b9a65a9a902bd6745234915a426df299e0b9b71ccf962683c85d0d05f87e1f"}
Jul 24 17:37:01.054910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:01.054853    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" podStartSLOduration=2.054835941 podStartE2EDuration="2.054835941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:01.052713475 +0000 UTC m=+1610.023845667" watchObservedRunningTime="2026-07-24 17:37:01.054835941 +0000 UTC m=+1610.025968547"
Jul 24 17:37:01.972359 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:01.972269    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:01.977870 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:01.977842    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:02.035616 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.035581    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:02.036710 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.036692    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:02.462418 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.462385    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"]
Jul 24 17:37:02.465913 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.465889    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:02.468414 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.468390    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:37:02.468556 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.468453    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8630be2b-serving-cert\""
Jul 24 17:37:02.468735 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.468715    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:37:02.468834 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.468782    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:37:02.478879 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.478669    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"]
Jul 24 17:37:02.545303 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.545267    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:02.545303 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.545308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxvw7\" (UniqueName: \"kubernetes.io/projected/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-kube-api-access-zxvw7\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:02.545522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.545382    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-openshift-service-ca\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:02.646435 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.646394    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-openshift-service-ca\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:02.646619 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.646505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:02.646619 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.646533    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxvw7\" (UniqueName: \"kubernetes.io/projected/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-kube-api-access-zxvw7\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:02.646691 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:02.646653    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-serving-cert: secret "maas-api-e2e-ait-8630be2b-serving-cert" not found
Jul 24 17:37:02.646759 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:02.646745    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls podName:a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:03.146719945 +0000 UTC m=+1612.117852107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls") pod "maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5" (UID: "a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722") : secret "maas-api-e2e-ait-8630be2b-serving-cert" not found
Jul 24 17:37:02.647197 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.647178    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-openshift-service-ca\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:02.658907 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:02.658872    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxvw7\" (UniqueName: \"kubernetes.io/projected/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-kube-api-access-zxvw7\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:03.042527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.042493    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"]
Jul 24 17:37:03.045890 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.045872    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:03.048781 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.048765    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8630be2b\""
Jul 24 17:37:03.060253 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.060227    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"]
Jul 24 17:37:03.083239 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.083204    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"]
Jul 24 17:37:03.086385 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.086369    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:03.088964 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.088943    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8630be2b-dockercfg-zzcz2\""
Jul 24 17:37:03.097168 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.097140    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"]
Jul 24 17:37:03.150049 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.150006    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:03.150253 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.150070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjtnl\" (UniqueName: \"kubernetes.io/projected/65e4a4b5-b164-48f1-856a-19c0cdc33776-kube-api-access-qjtnl\") pod \"payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4\" (UID: \"65e4a4b5-b164-48f1-856a-19c0cdc33776\") " pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:03.150253 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.150099    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm\" (UID: \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:03.150253 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.150153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf9mb\" (UniqueName: \"kubernetes.io/projected/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-kube-api-access-tf9mb\") pod \"payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm\" (UID: \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:03.150253 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.150194    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4a4b5-b164-48f1-856a-19c0cdc33776-plugins-config-volume\") pod \"payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4\" (UID: \"65e4a4b5-b164-48f1-856a-19c0cdc33776\") " pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:03.152661 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.152636    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls\") pod \"maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:03.251288 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.251243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf9mb\" (UniqueName: \"kubernetes.io/projected/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-kube-api-access-tf9mb\") pod \"payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm\" (UID: \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:03.251499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.251298    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4a4b5-b164-48f1-856a-19c0cdc33776-plugins-config-volume\") pod \"payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4\" (UID: \"65e4a4b5-b164-48f1-856a-19c0cdc33776\") " pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:03.251499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.251380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjtnl\" (UniqueName: \"kubernetes.io/projected/65e4a4b5-b164-48f1-856a-19c0cdc33776-kube-api-access-qjtnl\") pod \"payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4\" (UID: \"65e4a4b5-b164-48f1-856a-19c0cdc33776\") " pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:03.251499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.251408    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm\" (UID: \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:03.252152 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.252130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm\" (UID: \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:03.252322 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.252299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4a4b5-b164-48f1-856a-19c0cdc33776-plugins-config-volume\") pod \"payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4\" (UID: \"65e4a4b5-b164-48f1-856a-19c0cdc33776\") " pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:03.260401 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.260376    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjtnl\" (UniqueName: \"kubernetes.io/projected/65e4a4b5-b164-48f1-856a-19c0cdc33776-kube-api-access-qjtnl\") pod \"payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4\" (UID: \"65e4a4b5-b164-48f1-856a-19c0cdc33776\") " pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:03.260532 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.260453    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf9mb\" (UniqueName: \"kubernetes.io/projected/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-kube-api-access-tf9mb\") pod \"payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm\" (UID: \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:03.355736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.355646    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:03.381261 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.381012    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:03.397388 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.397238    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:03.534858 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.534777    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"]
Jul 24 17:37:03.537166 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:37:03.537098    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded1ae28_0bb7_4dbd_a0eb_078d7d087890.slice/crio-01e189be1145ae48d3d460d17b3552df96b89a780b5927c23aee9c5467be1ec0 WatchSource:0}: Error finding container 01e189be1145ae48d3d460d17b3552df96b89a780b5927c23aee9c5467be1ec0: Status 404 returned error can't find the container with id 01e189be1145ae48d3d460d17b3552df96b89a780b5927c23aee9c5467be1ec0
Jul 24 17:37:03.575157 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.575116    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"]
Jul 24 17:37:03.579754 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:37:03.579724    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda60d48d4_ca9a_4aeb_aaa2_1df2ea4b0722.slice/crio-59df1527510e9fca1fc492fe86d0669bf36f5b28ae49f9c25d10c07d251e353a WatchSource:0}: Error finding container 59df1527510e9fca1fc492fe86d0669bf36f5b28ae49f9c25d10c07d251e353a: Status 404 returned error can't find the container with id 59df1527510e9fca1fc492fe86d0669bf36f5b28ae49f9c25d10c07d251e353a
Jul 24 17:37:03.581535 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.581519    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:37:03.593312 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:03.593283    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"]
Jul 24 17:37:03.594873 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:37:03.594847    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e4a4b5_b164_48f1_856a_19c0cdc33776.slice/crio-eeb484f783dd0c2a048ea3d6e38ba550ff98c599fb91753d6dab55468b222ceb WatchSource:0}: Error finding container eeb484f783dd0c2a048ea3d6e38ba550ff98c599fb91753d6dab55468b222ceb: Status 404 returned error can't find the container with id eeb484f783dd0c2a048ea3d6e38ba550ff98c599fb91753d6dab55468b222ceb
Jul 24 17:37:04.045880 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.045839    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm" event={"ID":"ded1ae28-0bb7-4dbd-a0eb-078d7d087890","Type":"ContainerStarted","Data":"b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b"}
Jul 24 17:37:04.046393 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.046006    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:04.046393 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.046064    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm" event={"ID":"ded1ae28-0bb7-4dbd-a0eb-078d7d087890","Type":"ContainerStarted","Data":"01e189be1145ae48d3d460d17b3552df96b89a780b5927c23aee9c5467be1ec0"}
Jul 24 17:37:04.047399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.047375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4" event={"ID":"65e4a4b5-b164-48f1-856a-19c0cdc33776","Type":"ContainerStarted","Data":"387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04"}
Jul 24 17:37:04.047528 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.047419    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4" event={"ID":"65e4a4b5-b164-48f1-856a-19c0cdc33776","Type":"ContainerStarted","Data":"eeb484f783dd0c2a048ea3d6e38ba550ff98c599fb91753d6dab55468b222ceb"}
Jul 24 17:37:04.047528 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.047445    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:04.048488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.048466    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5" event={"ID":"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722","Type":"ContainerStarted","Data":"59df1527510e9fca1fc492fe86d0669bf36f5b28ae49f9c25d10c07d251e353a"}
Jul 24 17:37:04.062769 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.062719    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm" podStartSLOduration=1.06270441 podStartE2EDuration="1.06270441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:04.061268705 +0000 UTC m=+1613.032400886" watchObservedRunningTime="2026-07-24 17:37:04.06270441 +0000 UTC m=+1613.033836591"
Jul 24 17:37:04.078128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:04.078074    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4" podStartSLOduration=1.078056305 podStartE2EDuration="1.078056305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:04.077155563 +0000 UTC m=+1613.048287745" watchObservedRunningTime="2026-07-24 17:37:04.078056305 +0000 UTC m=+1613.049188485"
Jul 24 17:37:06.060720 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:06.060686    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5" event={"ID":"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722","Type":"ContainerStarted","Data":"c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9"}
Jul 24 17:37:06.061249 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:06.060750    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:06.079486 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:06.079435    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5" podStartSLOduration=2.490018924 podStartE2EDuration="4.079416643s" totalImagesPullingTime="1.589397719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:06.078031335 +0000 UTC m=+1615.049163509" watchObservedRunningTime="2026-07-24 17:37:06.079416643 +0000 UTC m=+1615.050548825"
Jul 24 17:37:12.074287 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:12.074253    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:15.055963 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:15.055927    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:15.056434 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:15.056040    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:17.358891 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:17.358838    2562 patch_prober.go:28] interesting pod/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": read tcp 10.133.0.2:41116->10.133.0.28:15021: read: connection reset by peer" start-of-body=
Jul 24 17:37:17.359329 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:17.358898    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": read tcp 10.133.0.2:41116->10.133.0.28:15021: read: connection reset by peer"
Jul 24 17:37:18.115081 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:18.115046    2562 generic.go:356] "Generic (PLEG): container finished" podID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerID="4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab" exitCode=0
Jul 24 17:37:18.115288 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:18.115048    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" event={"ID":"78289f81-c7a1-46f2-b070-65d05acd3fd4","Type":"ContainerDied","Data":"4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab"}
Jul 24 17:37:18.115484 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:18.115469    2562 scope.go:117] "RemoveContainer" containerID="4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab"
Jul 24 17:37:18.116492 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:18.116464    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:18.116548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:18.116538    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:18.116596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:18.116584    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:19.121295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:19.121260    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" event={"ID":"78289f81-c7a1-46f2-b070-65d05acd3fd4","Type":"ContainerStarted","Data":"48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536"}
Jul 24 17:37:19.972173 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:19.972073    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:19.972412 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:19.972381    2562 patch_prober.go:28] interesting pod/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Jul 24 17:37:19.972498 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:19.972447    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Jul 24 17:37:20.972397 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:20.971922    2562 patch_prober.go:28] interesting pod/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Jul 24 17:37:20.972397 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:20.971985    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Jul 24 17:37:21.972853 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:21.972806    2562 patch_prober.go:28] interesting pod/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body=
Jul 24 17:37:21.973378 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:21.972890    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused"
Jul 24 17:37:22.975790 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:22.975763    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:22.976203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:22.976038    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:22.976677 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:22.976661    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:38.444965 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.444920    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"]
Jul 24 17:37:38.445595 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.445181    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5" podUID="a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" containerName="maas-api" containerID="cri-o://c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9" gracePeriod=30
Jul 24 17:37:38.493339 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.493261    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"]
Jul 24 17:37:38.493630 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.493599    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4" podUID="65e4a4b5-b164-48f1-856a-19c0cdc33776" containerName="payload-processing" containerID="cri-o://387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04" gracePeriod=30
Jul 24 17:37:38.505561 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.504298    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"]
Jul 24 17:37:38.505561 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.504607    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm" podUID="ded1ae28-0bb7-4dbd-a0eb-078d7d087890" containerName="payload-pre-processing" containerID="cri-o://b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b" gracePeriod=30
Jul 24 17:37:38.774478 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.774451    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:38.806483 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.806460    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:38.811469 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.811442    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:38.868818 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.868787    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-openshift-service-ca\") pod \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") "
Jul 24 17:37:38.869011 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.868836    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls\") pod \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") "
Jul 24 17:37:38.869011 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.868855    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxvw7\" (UniqueName: \"kubernetes.io/projected/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-kube-api-access-zxvw7\") pod \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\" (UID: \"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722\") "
Jul 24 17:37:38.869252 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.869227    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" (UID: "a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:37:38.871185 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.871164    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" (UID: "a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:37:38.871250 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.871202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-kube-api-access-zxvw7" (OuterVolumeSpecName: "kube-api-access-zxvw7") pod "a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" (UID: "a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722"). InnerVolumeSpecName "kube-api-access-zxvw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:37:38.969611 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.969573    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjtnl\" (UniqueName: \"kubernetes.io/projected/65e4a4b5-b164-48f1-856a-19c0cdc33776-kube-api-access-qjtnl\") pod \"65e4a4b5-b164-48f1-856a-19c0cdc33776\" (UID: \"65e4a4b5-b164-48f1-856a-19c0cdc33776\") "
Jul 24 17:37:38.969611 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.969634    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4a4b5-b164-48f1-856a-19c0cdc33776-plugins-config-volume\") pod \"65e4a4b5-b164-48f1-856a-19c0cdc33776\" (UID: \"65e4a4b5-b164-48f1-856a-19c0cdc33776\") "
Jul 24 17:37:38.969851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.969678    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf9mb\" (UniqueName: \"kubernetes.io/projected/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-kube-api-access-tf9mb\") pod \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\" (UID: \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\") "
Jul 24 17:37:38.969851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.969698    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-plugins-config-volume\") pod \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\" (UID: \"ded1ae28-0bb7-4dbd-a0eb-078d7d087890\") "
Jul 24 17:37:38.969967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.969853    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:38.969967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.969867    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:38.969967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.969876    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxvw7\" (UniqueName: \"kubernetes.io/projected/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722-kube-api-access-zxvw7\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:38.970149 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.970120    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65e4a4b5-b164-48f1-856a-19c0cdc33776-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65e4a4b5-b164-48f1-856a-19c0cdc33776" (UID: "65e4a4b5-b164-48f1-856a-19c0cdc33776"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:37:38.970196 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.970141    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ded1ae28-0bb7-4dbd-a0eb-078d7d087890" (UID: "ded1ae28-0bb7-4dbd-a0eb-078d7d087890"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:37:38.971790 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.971766    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65e4a4b5-b164-48f1-856a-19c0cdc33776-kube-api-access-qjtnl" (OuterVolumeSpecName: "kube-api-access-qjtnl") pod "65e4a4b5-b164-48f1-856a-19c0cdc33776" (UID: "65e4a4b5-b164-48f1-856a-19c0cdc33776"). InnerVolumeSpecName "kube-api-access-qjtnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:37:38.971863 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:38.971843    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-kube-api-access-tf9mb" (OuterVolumeSpecName: "kube-api-access-tf9mb") pod "ded1ae28-0bb7-4dbd-a0eb-078d7d087890" (UID: "ded1ae28-0bb7-4dbd-a0eb-078d7d087890"). InnerVolumeSpecName "kube-api-access-tf9mb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:37:39.070701 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.070629    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf9mb\" (UniqueName: \"kubernetes.io/projected/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-kube-api-access-tf9mb\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:39.070701 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.070653    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ded1ae28-0bb7-4dbd-a0eb-078d7d087890-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:39.070701 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.070664    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjtnl\" (UniqueName: \"kubernetes.io/projected/65e4a4b5-b164-48f1-856a-19c0cdc33776-kube-api-access-qjtnl\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:39.070701 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.070673    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65e4a4b5-b164-48f1-856a-19c0cdc33776-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:39.209510 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.209471    2562 generic.go:356] "Generic (PLEG): container finished" podID="a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" containerID="c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9" exitCode=0
Jul 24 17:37:39.209691 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.209521    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"
Jul 24 17:37:39.209691 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.209520    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5" event={"ID":"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722","Type":"ContainerDied","Data":"c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9"}
Jul 24 17:37:39.209691 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.209576    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5" event={"ID":"a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722","Type":"ContainerDied","Data":"59df1527510e9fca1fc492fe86d0669bf36f5b28ae49f9c25d10c07d251e353a"}
Jul 24 17:37:39.209691 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.209606    2562 scope.go:117] "RemoveContainer" containerID="c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9"
Jul 24 17:37:39.210891 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.210870    2562 generic.go:356] "Generic (PLEG): container finished" podID="ded1ae28-0bb7-4dbd-a0eb-078d7d087890" containerID="b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b" exitCode=0
Jul 24 17:37:39.210998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.210932    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"
Jul 24 17:37:39.210998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.210957    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm" event={"ID":"ded1ae28-0bb7-4dbd-a0eb-078d7d087890","Type":"ContainerDied","Data":"b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b"}
Jul 24 17:37:39.211142 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.211002    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm" event={"ID":"ded1ae28-0bb7-4dbd-a0eb-078d7d087890","Type":"ContainerDied","Data":"01e189be1145ae48d3d460d17b3552df96b89a780b5927c23aee9c5467be1ec0"}
Jul 24 17:37:39.212376 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.212344    2562 generic.go:356] "Generic (PLEG): container finished" podID="65e4a4b5-b164-48f1-856a-19c0cdc33776" containerID="387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04" exitCode=0
Jul 24 17:37:39.212479 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.212381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4" event={"ID":"65e4a4b5-b164-48f1-856a-19c0cdc33776","Type":"ContainerDied","Data":"387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04"}
Jul 24 17:37:39.212479 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.212389    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"
Jul 24 17:37:39.212479 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.212415    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4" event={"ID":"65e4a4b5-b164-48f1-856a-19c0cdc33776","Type":"ContainerDied","Data":"eeb484f783dd0c2a048ea3d6e38ba550ff98c599fb91753d6dab55468b222ceb"}
Jul 24 17:37:39.220302 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.220241    2562 scope.go:117] "RemoveContainer" containerID="c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9"
Jul 24 17:37:39.220554 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:39.220520    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9\": container with ID starting with c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9 not found: ID does not exist" containerID="c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9"
Jul 24 17:37:39.220661 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.220563    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9"} err="failed to get container status \"c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9\": rpc error: code = NotFound desc = could not find container \"c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9\": container with ID starting with c62ebca782a0aa08bdc89708b6ded3567924325d4bbc495e56b2d5d82c3db0a9 not found: ID does not exist"
Jul 24 17:37:39.220661 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.220585    2562 scope.go:117] "RemoveContainer" containerID="b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b"
Jul 24 17:37:39.228800 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.228784    2562 scope.go:117] "RemoveContainer" containerID="b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b"
Jul 24 17:37:39.229075 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:39.229057    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b\": container with ID starting with b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b not found: ID does not exist" containerID="b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b"
Jul 24 17:37:39.229139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.229081    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b"} err="failed to get container status \"b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b\": rpc error: code = NotFound desc = could not find container \"b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b\": container with ID starting with b7458f1e40e395ffdd2dc9824fec8ef1fe5bc9595b3f7df82d85ee4de034f00b not found: ID does not exist"
Jul 24 17:37:39.229139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.229097    2562 scope.go:117] "RemoveContainer" containerID="387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04"
Jul 24 17:37:39.237736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.237705    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"]
Jul 24 17:37:39.237838 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.237711    2562 scope.go:117] "RemoveContainer" containerID="387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04"
Jul 24 17:37:39.238059 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:39.238042    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04\": container with ID starting with 387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04 not found: ID does not exist" containerID="387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04"
Jul 24 17:37:39.238128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.238070    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04"} err="failed to get container status \"387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04\": rpc error: code = NotFound desc = could not find container \"387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04\": container with ID starting with 387a8900eea02655d4544b7ca089efb93ecf13adaaa04d8accd1b3a6c2b96a04 not found: ID does not exist"
Jul 24 17:37:39.242981 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.242956    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8630be2b-7cf4c44655-m4xt5"]
Jul 24 17:37:39.252905 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.252880    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"]
Jul 24 17:37:39.254998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.254978    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8630be2b-58d8db48c6-8mvc4"]
Jul 24 17:37:39.265212 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.265189    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"]
Jul 24 17:37:39.270468 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.270450    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8630be2b-85fdf95848-vz2sm"]
Jul 24 17:37:39.505755 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.505721    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65e4a4b5-b164-48f1-856a-19c0cdc33776" path="/var/lib/kubelet/pods/65e4a4b5-b164-48f1-856a-19c0cdc33776/volumes"
Jul 24 17:37:39.506140 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.506120    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" path="/var/lib/kubelet/pods/a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722/volumes"
Jul 24 17:37:39.506423 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:39.506410    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ded1ae28-0bb7-4dbd-a0eb-078d7d087890" path="/var/lib/kubelet/pods/ded1ae28-0bb7-4dbd-a0eb-078d7d087890/volumes"
Jul 24 17:37:43.214559 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:43.214523    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"]
Jul 24 17:37:43.215045 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:43.214790    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy" containerID="cri-o://48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536" gracePeriod=30
Jul 24 17:37:44.271729 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.271699    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"]
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272070    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" containerName="maas-api"
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272084    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" containerName="maas-api"
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272095    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65e4a4b5-b164-48f1-856a-19c0cdc33776" containerName="payload-processing"
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272103    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="65e4a4b5-b164-48f1-856a-19c0cdc33776" containerName="payload-processing"
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272121    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ded1ae28-0bb7-4dbd-a0eb-078d7d087890" containerName="payload-pre-processing"
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272127    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ded1ae28-0bb7-4dbd-a0eb-078d7d087890" containerName="payload-pre-processing"
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272181    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="65e4a4b5-b164-48f1-856a-19c0cdc33776" containerName="payload-processing"
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272192    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ded1ae28-0bb7-4dbd-a0eb-078d7d087890" containerName="payload-pre-processing"
Jul 24 17:37:44.272208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.272200    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a60d48d4-ca9a-4aeb-aaa2-1df2ea4b0722" containerName="maas-api"
Jul 24 17:37:44.274947 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.274916    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.277591 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.277571    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f431f60c-gw-openshift-default-dockercfg-qx5th\""
Jul 24 17:37:44.287084 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.287062    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"]
Jul 24 17:37:44.315643 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-token\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.315826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315648    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-podinfo\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.315826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315723    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-certs\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.315826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istiod-ca-cert\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.315826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-socket\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.315826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-credential-socket\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.315826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-data\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.316133 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfzh9\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-kube-api-access-gfzh9\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.316133 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.315857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-envoy\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzh9\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-kube-api-access-gfzh9\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416295    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-envoy\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416503 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-token\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416503 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416484    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-podinfo\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416617 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416600    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-certs\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416660 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istiod-ca-cert\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416711 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416679    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-socket\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416761 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-credential-socket\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.416761 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416732    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-data\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.417001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.416977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-certs\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.417232 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.417051    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-data\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.417360 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.417251    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-socket\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.417478 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.417365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-credential-socket\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.417478 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.417421    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istiod-ca-cert\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.418683 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.418664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-envoy\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.419164 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.419147    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-podinfo\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.424609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.424583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-token\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.424744 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.424724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfzh9\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-kube-api-access-gfzh9\") pod \"e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") " pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.585978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.585891    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:44.717571 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.717543    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"]
Jul 24 17:37:44.719156 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:37:44.719131    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a1042c_64e6_4eec_aa91_54d1e67b8e6c.slice/crio-8f7cecb706c550455528c8ac401e93c6e6337eea0cd8dcd5d9d2b2c772433ff3 WatchSource:0}: Error finding container 8f7cecb706c550455528c8ac401e93c6e6337eea0cd8dcd5d9d2b2c772433ff3: Status 404 returned error can't find the container with id 8f7cecb706c550455528c8ac401e93c6e6337eea0cd8dcd5d9d2b2c772433ff3
Jul 24 17:37:44.721177 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.721146    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:44.721245 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.721211    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:44.721245 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:44.721239    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:37:45.243245 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:45.243214    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" event={"ID":"56a1042c-64e6-4eec-aa91-54d1e67b8e6c","Type":"ContainerStarted","Data":"4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152"}
Jul 24 17:37:45.243434 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:45.243266    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" event={"ID":"56a1042c-64e6-4eec-aa91-54d1e67b8e6c","Type":"ContainerStarted","Data":"8f7cecb706c550455528c8ac401e93c6e6337eea0cd8dcd5d9d2b2c772433ff3"}
Jul 24 17:37:45.268116 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:45.268053    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" podStartSLOduration=1.268031655 podStartE2EDuration="1.268031655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:45.265530685 +0000 UTC m=+1654.236662867" watchObservedRunningTime="2026-07-24 17:37:45.268031655 +0000 UTC m=+1654.239163828"
Jul 24 17:37:45.586297 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:45.586208    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:45.591230 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:45.591206    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:46.247378 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.247342    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:46.248257 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.248237    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:37:46.814461 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.814422    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"]
Jul 24 17:37:46.817837 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.817813    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:46.820651 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.820625    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-f431f60c-serving-cert\""
Jul 24 17:37:46.820809 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.820681    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:37:46.821946 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.821926    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:37:46.822071 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.822006    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:37:46.826644 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.826623    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"]
Jul 24 17:37:46.839059 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.839011    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5psr\" (UniqueName: \"kubernetes.io/projected/2470a645-deb4-4505-8937-5b5bb75df8d2-kube-api-access-h5psr\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:46.839179 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.839079    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2470a645-deb4-4505-8937-5b5bb75df8d2-openshift-service-ca\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:46.839231 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.839213    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:46.940647 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.940615    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:46.940821 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.940676    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5psr\" (UniqueName: \"kubernetes.io/projected/2470a645-deb4-4505-8937-5b5bb75df8d2-kube-api-access-h5psr\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:46.940821 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:46.940771    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-serving-cert: secret "maas-api-e2e-migrate-f431f60c-serving-cert" not found
Jul 24 17:37:46.940821 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.940786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2470a645-deb4-4505-8937-5b5bb75df8d2-openshift-service-ca\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:46.940960 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:46.940842    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls podName:2470a645-deb4-4505-8937-5b5bb75df8d2 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:47.4408201 +0000 UTC m=+1656.411952269 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls") pod "maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj" (UID: "2470a645-deb4-4505-8937-5b5bb75df8d2") : secret "maas-api-e2e-migrate-f431f60c-serving-cert" not found
Jul 24 17:37:46.941502 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.941482    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2470a645-deb4-4505-8937-5b5bb75df8d2-openshift-service-ca\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:46.950641 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:46.950611    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5psr\" (UniqueName: \"kubernetes.io/projected/2470a645-deb4-4505-8937-5b5bb75df8d2-kube-api-access-h5psr\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:47.439610 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.439575    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"]
Jul 24 17:37:47.442864 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.442849    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:47.445842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.445807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:47.445965 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.445861    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-f431f60c\""
Jul 24 17:37:47.445965 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.445877    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-f431f60c-dockercfg-l58qh\""
Jul 24 17:37:47.448621 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.448601    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls\") pod \"maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:47.454005 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.453981    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"]
Jul 24 17:37:47.466348 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.466326    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"]
Jul 24 17:37:47.469527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.469508    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:47.479460 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.479437    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"]
Jul 24 17:37:47.547284 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.547251    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv29f\" (UniqueName: \"kubernetes.io/projected/d62711be-165d-4942-96d0-0c3d54f1eae2-kube-api-access-xv29f\") pod \"payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v\" (UID: \"d62711be-165d-4942-96d0-0c3d54f1eae2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:47.547462 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.547401    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c803fdc-7c4a-4332-a6f2-128251ffa074-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc\" (UID: \"0c803fdc-7c4a-4332-a6f2-128251ffa074\") " pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:47.547462 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.547435    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t49x\" (UniqueName: \"kubernetes.io/projected/0c803fdc-7c4a-4332-a6f2-128251ffa074-kube-api-access-5t49x\") pod \"payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc\" (UID: \"0c803fdc-7c4a-4332-a6f2-128251ffa074\") " pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:47.547548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.547497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d62711be-165d-4942-96d0-0c3d54f1eae2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v\" (UID: \"d62711be-165d-4942-96d0-0c3d54f1eae2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:47.648713 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.648681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c803fdc-7c4a-4332-a6f2-128251ffa074-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc\" (UID: \"0c803fdc-7c4a-4332-a6f2-128251ffa074\") " pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:47.648713 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.648717    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t49x\" (UniqueName: \"kubernetes.io/projected/0c803fdc-7c4a-4332-a6f2-128251ffa074-kube-api-access-5t49x\") pod \"payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc\" (UID: \"0c803fdc-7c4a-4332-a6f2-128251ffa074\") " pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:47.648884 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.648753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d62711be-165d-4942-96d0-0c3d54f1eae2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v\" (UID: \"d62711be-165d-4942-96d0-0c3d54f1eae2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:47.648939 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.648925    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv29f\" (UniqueName: \"kubernetes.io/projected/d62711be-165d-4942-96d0-0c3d54f1eae2-kube-api-access-xv29f\") pod \"payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v\" (UID: \"d62711be-165d-4942-96d0-0c3d54f1eae2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:47.649520 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.649496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d62711be-165d-4942-96d0-0c3d54f1eae2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v\" (UID: \"d62711be-165d-4942-96d0-0c3d54f1eae2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:47.649641 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.649578    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c803fdc-7c4a-4332-a6f2-128251ffa074-plugins-config-volume\") pod \"payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc\" (UID: \"0c803fdc-7c4a-4332-a6f2-128251ffa074\") " pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:47.657217 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.657195    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t49x\" (UniqueName: \"kubernetes.io/projected/0c803fdc-7c4a-4332-a6f2-128251ffa074-kube-api-access-5t49x\") pod \"payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc\" (UID: \"0c803fdc-7c4a-4332-a6f2-128251ffa074\") " pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:47.657314 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.657237    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv29f\" (UniqueName: \"kubernetes.io/projected/d62711be-165d-4942-96d0-0c3d54f1eae2-kube-api-access-xv29f\") pod \"payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v\" (UID: \"d62711be-165d-4942-96d0-0c3d54f1eae2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:47.731174 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.731144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:47.762665 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.762630    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:47.779921 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.779848    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:47.919957 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.919927    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"]
Jul 24 17:37:47.921997 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:37:47.921965    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2470a645_deb4_4505_8937_5b5bb75df8d2.slice/crio-6f75b04c3c298dab612f94aaf080e24edfe9867c1e634472e7ae9071f1e8ed52 WatchSource:0}: Error finding container 6f75b04c3c298dab612f94aaf080e24edfe9867c1e634472e7ae9071f1e8ed52: Status 404 returned error can't find the container with id 6f75b04c3c298dab612f94aaf080e24edfe9867c1e634472e7ae9071f1e8ed52
Jul 24 17:37:47.953844 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.953769    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"]
Jul 24 17:37:47.955210 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:37:47.955168    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62711be_165d_4942_96d0_0c3d54f1eae2.slice/crio-bdcacc9187805aaa316feca2c04508bda9bf1ca294dc56dbfdf1bafb70245f98 WatchSource:0}: Error finding container bdcacc9187805aaa316feca2c04508bda9bf1ca294dc56dbfdf1bafb70245f98: Status 404 returned error can't find the container with id bdcacc9187805aaa316feca2c04508bda9bf1ca294dc56dbfdf1bafb70245f98
Jul 24 17:37:47.977792 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:47.977761    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"]
Jul 24 17:37:47.980314 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:37:47.980273    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c803fdc_7c4a_4332_a6f2_128251ffa074.slice/crio-1834d48f78b125c60284f6ca96c95b841f4ab1872b97488f211f4a2f4cf45d7c WatchSource:0}: Error finding container 1834d48f78b125c60284f6ca96c95b841f4ab1872b97488f211f4a2f4cf45d7c: Status 404 returned error can't find the container with id 1834d48f78b125c60284f6ca96c95b841f4ab1872b97488f211f4a2f4cf45d7c
Jul 24 17:37:48.257974 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.257872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj" event={"ID":"2470a645-deb4-4505-8937-5b5bb75df8d2","Type":"ContainerStarted","Data":"6f75b04c3c298dab612f94aaf080e24edfe9867c1e634472e7ae9071f1e8ed52"}
Jul 24 17:37:48.259696 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.259663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc" event={"ID":"0c803fdc-7c4a-4332-a6f2-128251ffa074","Type":"ContainerStarted","Data":"710c6eba2b9908114708d49bb658e0afd48be5c06509b67cd9894c01e2eddb5f"}
Jul 24 17:37:48.259846 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.259731    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc" event={"ID":"0c803fdc-7c4a-4332-a6f2-128251ffa074","Type":"ContainerStarted","Data":"1834d48f78b125c60284f6ca96c95b841f4ab1872b97488f211f4a2f4cf45d7c"}
Jul 24 17:37:48.259846 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.259764    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:48.261475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.261450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v" event={"ID":"d62711be-165d-4942-96d0-0c3d54f1eae2","Type":"ContainerStarted","Data":"2a7d5d951e610f2f09fdc57fdad10848197f617b4376e9cffd80e1ced82c6203"}
Jul 24 17:37:48.261610 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.261496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v" event={"ID":"d62711be-165d-4942-96d0-0c3d54f1eae2","Type":"ContainerStarted","Data":"bdcacc9187805aaa316feca2c04508bda9bf1ca294dc56dbfdf1bafb70245f98"}
Jul 24 17:37:48.261610 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.261548    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:37:48.278189 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.278142    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc" podStartSLOduration=1.2781253590000001 podStartE2EDuration="1.278125359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:48.275660736 +0000 UTC m=+1657.246792930" watchObservedRunningTime="2026-07-24 17:37:48.278125359 +0000 UTC m=+1657.249257539"
Jul 24 17:37:48.291413 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.291360    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v" podStartSLOduration=1.291342108 podStartE2EDuration="1.291342108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:48.290952911 +0000 UTC m=+1657.262085116" watchObservedRunningTime="2026-07-24 17:37:48.291342108 +0000 UTC m=+1657.262474290"
Jul 24 17:37:48.467283 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.467261    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:48.555853 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.555774    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-token\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.555853 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.555817    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-socket\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.555853 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.555851    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-certs\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.556175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.555874    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-data\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.556175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.555936    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9s75\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-kube-api-access-t9s75\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.556175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.555969    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-credential-socket\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.556175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.555993    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78289f81-c7a1-46f2-b070-65d05acd3fd4-istiod-ca-cert\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.556175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.556060    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-podinfo\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.556175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.556090    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-envoy\") pod \"78289f81-c7a1-46f2-b070-65d05acd3fd4\" (UID: \"78289f81-c7a1-46f2-b070-65d05acd3fd4\") "
Jul 24 17:37:48.556175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.556159    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:37:48.556568 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.556425    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:48.556964 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.556697    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:37:48.556964 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.556884    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:37:48.557576 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.557541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78289f81-c7a1-46f2-b070-65d05acd3fd4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:37:48.558200 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.558174    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-data" (OuterVolumeSpecName: "istio-data") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:37:48.558582 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.558549    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:37:48.558916 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.558878    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-token" (OuterVolumeSpecName: "istio-token") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:37:48.559290 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.559265    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:37:48.560567 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.560541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-kube-api-access-t9s75" (OuterVolumeSpecName: "kube-api-access-t9s75") pod "78289f81-c7a1-46f2-b070-65d05acd3fd4" (UID: "78289f81-c7a1-46f2-b070-65d05acd3fd4"). InnerVolumeSpecName "kube-api-access-t9s75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:37:48.656907 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.656868    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9s75\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-kube-api-access-t9s75\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:48.656907 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.656899    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:48.656907 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.656909    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78289f81-c7a1-46f2-b070-65d05acd3fd4-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:48.657203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.656919    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:48.657203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.656927    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:48.657203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.656934    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:48.657203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.656941    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:48.657203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:48.656948    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78289f81-c7a1-46f2-b070-65d05acd3fd4-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:37:49.268495 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.268388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj" event={"ID":"2470a645-deb4-4505-8937-5b5bb75df8d2","Type":"ContainerStarted","Data":"36b6007a5a854e3823d9e74ece49e80c952a794e12289616e03e0653df71af46"}
Jul 24 17:37:49.268495 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.268486    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:49.270172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.270147    2562 generic.go:356] "Generic (PLEG): container finished" podID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerID="48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536" exitCode=0
Jul 24 17:37:49.270319 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.270208    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"
Jul 24 17:37:49.270319 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.270228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" event={"ID":"78289f81-c7a1-46f2-b070-65d05acd3fd4","Type":"ContainerDied","Data":"48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536"}
Jul 24 17:37:49.270319 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.270279    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv" event={"ID":"78289f81-c7a1-46f2-b070-65d05acd3fd4","Type":"ContainerDied","Data":"38b9a65a9a902bd6745234915a426df299e0b9b71ccf962683c85d0d05f87e1f"}
Jul 24 17:37:49.270423 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.270311    2562 scope.go:117] "RemoveContainer" containerID="48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536"
Jul 24 17:37:49.279389 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.279371    2562 scope.go:117] "RemoveContainer" containerID="4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab"
Jul 24 17:37:49.286603 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.286552    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj" podStartSLOduration=2.828653387 podStartE2EDuration="3.286533626s" totalImagesPullingTime="457.880239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:49.28491329 +0000 UTC m=+1658.256045473" watchObservedRunningTime="2026-07-24 17:37:49.286533626 +0000 UTC m=+1658.257665808"
Jul 24 17:37:49.288079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.288058    2562 scope.go:117] "RemoveContainer" containerID="48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536"
Jul 24 17:37:49.288376 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:49.288352    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536\": container with ID starting with 48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536 not found: ID does not exist" containerID="48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536"
Jul 24 17:37:49.288454 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.288384    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536"} err="failed to get container status \"48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536\": rpc error: code = NotFound desc = could not find container \"48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536\": container with ID starting with 48551a9d56bd8ed3523b4b882f676e510060d9dc20c9f0df9fa8cd1b6dbb9536 not found: ID does not exist"
Jul 24 17:37:49.288454 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.288401    2562 scope.go:117] "RemoveContainer" containerID="4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab"
Jul 24 17:37:49.288686 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:37:49.288662    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab\": container with ID starting with 4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab not found: ID does not exist" containerID="4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab"
Jul 24 17:37:49.288755 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.288690    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab"} err="failed to get container status \"4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab\": rpc error: code = NotFound desc = could not find container \"4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab\": container with ID starting with 4bd28aea235bd3b9c7c18c371961f35bf37c2673ff1046e1e641c2806e469cab not found: ID does not exist"
Jul 24 17:37:49.298631 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.298606    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"]
Jul 24 17:37:49.301909 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.301887    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8630be2b-openshift-default-64b59dcd96-qxcpv"]
Jul 24 17:37:49.505423 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:49.505390    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" path="/var/lib/kubelet/pods/78289f81-c7a1-46f2-b070-65d05acd3fd4/volumes"
Jul 24 17:37:55.281480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:55.281453    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:37:59.272557 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:59.272520    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:37:59.273070 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:37:59.272594    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:38:01.249422 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:01.249385    2562 patch_prober.go:28] interesting pod/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 24 17:38:01.249872 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:01.249453    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 24 17:38:01.331477 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:01.331443    2562 generic.go:356] "Generic (PLEG): container finished" podID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerID="4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152" exitCode=0
Jul 24 17:38:01.331648 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:01.331514    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" event={"ID":"56a1042c-64e6-4eec-aa91-54d1e67b8e6c","Type":"ContainerDied","Data":"4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152"}
Jul 24 17:38:01.331928 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:01.331912    2562 scope.go:117] "RemoveContainer" containerID="4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152"
Jul 24 17:38:01.332904 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:01.332879    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:01.332969 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:01.332947    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:01.333005 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:01.332979    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:02.337852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:02.337817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" event={"ID":"56a1042c-64e6-4eec-aa91-54d1e67b8e6c","Type":"ContainerStarted","Data":"810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9"}
Jul 24 17:38:02.586611 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:02.586575    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:38:02.586795 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:02.586775    2562 patch_prober.go:28] interesting pod/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 24 17:38:02.586845 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:02.586824    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 24 17:38:03.586293 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:03.586256    2562 patch_prober.go:28] interesting pod/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 24 17:38:03.586762 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:03.586321    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 24 17:38:04.586375 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:04.586335    2562 patch_prober.go:28] interesting pod/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 24 17:38:04.586786 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:04.586392    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 24 17:38:05.586745 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:05.586701    2562 patch_prober.go:28] interesting pod/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Jul 24 17:38:05.587164 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:05.586762    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Jul 24 17:38:06.589859 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:06.589834    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:38:06.590357 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:06.590337    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:38:06.591064 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:06.591046    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:38:22.162657 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.162624    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"]
Jul 24 17:38:22.163193 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.162835    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj" podUID="2470a645-deb4-4505-8937-5b5bb75df8d2" containerName="maas-api" containerID="cri-o://36b6007a5a854e3823d9e74ece49e80c952a794e12289616e03e0653df71af46" gracePeriod=30
Jul 24 17:38:22.204148 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.204118    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"]
Jul 24 17:38:22.204340 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.204310    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc" podUID="0c803fdc-7c4a-4332-a6f2-128251ffa074" containerName="payload-processing" containerID="cri-o://710c6eba2b9908114708d49bb658e0afd48be5c06509b67cd9894c01e2eddb5f" gracePeriod=30
Jul 24 17:38:22.209775 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.209196    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"]
Jul 24 17:38:22.209775 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.209425    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v" podUID="d62711be-165d-4942-96d0-0c3d54f1eae2" containerName="payload-pre-processing" containerID="cri-o://2a7d5d951e610f2f09fdc57fdad10848197f617b4376e9cffd80e1ced82c6203" gracePeriod=30
Jul 24 17:38:22.437255 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.436967    2562 generic.go:356] "Generic (PLEG): container finished" podID="d62711be-165d-4942-96d0-0c3d54f1eae2" containerID="2a7d5d951e610f2f09fdc57fdad10848197f617b4376e9cffd80e1ced82c6203" exitCode=0
Jul 24 17:38:22.437255 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.437124    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v" event={"ID":"d62711be-165d-4942-96d0-0c3d54f1eae2","Type":"ContainerDied","Data":"2a7d5d951e610f2f09fdc57fdad10848197f617b4376e9cffd80e1ced82c6203"}
Jul 24 17:38:22.439621 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.439595    2562 generic.go:356] "Generic (PLEG): container finished" podID="2470a645-deb4-4505-8937-5b5bb75df8d2" containerID="36b6007a5a854e3823d9e74ece49e80c952a794e12289616e03e0653df71af46" exitCode=0
Jul 24 17:38:22.439769 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.439646    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj" event={"ID":"2470a645-deb4-4505-8937-5b5bb75df8d2","Type":"ContainerDied","Data":"36b6007a5a854e3823d9e74ece49e80c952a794e12289616e03e0653df71af46"}
Jul 24 17:38:22.441349 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.441247    2562 generic.go:356] "Generic (PLEG): container finished" podID="0c803fdc-7c4a-4332-a6f2-128251ffa074" containerID="710c6eba2b9908114708d49bb658e0afd48be5c06509b67cd9894c01e2eddb5f" exitCode=0
Jul 24 17:38:22.441349 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.441292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc" event={"ID":"0c803fdc-7c4a-4332-a6f2-128251ffa074","Type":"ContainerDied","Data":"710c6eba2b9908114708d49bb658e0afd48be5c06509b67cd9894c01e2eddb5f"}
Jul 24 17:38:22.493335 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.493308    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:38:22.521086 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.521059    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:38:22.524214 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.524189    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:38:22.649302 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649259    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2470a645-deb4-4505-8937-5b5bb75df8d2-openshift-service-ca\") pod \"2470a645-deb4-4505-8937-5b5bb75df8d2\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") "
Jul 24 17:38:22.649508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649316    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d62711be-165d-4942-96d0-0c3d54f1eae2-plugins-config-volume\") pod \"d62711be-165d-4942-96d0-0c3d54f1eae2\" (UID: \"d62711be-165d-4942-96d0-0c3d54f1eae2\") "
Jul 24 17:38:22.649508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649347    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t49x\" (UniqueName: \"kubernetes.io/projected/0c803fdc-7c4a-4332-a6f2-128251ffa074-kube-api-access-5t49x\") pod \"0c803fdc-7c4a-4332-a6f2-128251ffa074\" (UID: \"0c803fdc-7c4a-4332-a6f2-128251ffa074\") "
Jul 24 17:38:22.649508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649394    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c803fdc-7c4a-4332-a6f2-128251ffa074-plugins-config-volume\") pod \"0c803fdc-7c4a-4332-a6f2-128251ffa074\" (UID: \"0c803fdc-7c4a-4332-a6f2-128251ffa074\") "
Jul 24 17:38:22.649508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649446    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv29f\" (UniqueName: \"kubernetes.io/projected/d62711be-165d-4942-96d0-0c3d54f1eae2-kube-api-access-xv29f\") pod \"d62711be-165d-4942-96d0-0c3d54f1eae2\" (UID: \"d62711be-165d-4942-96d0-0c3d54f1eae2\") "
Jul 24 17:38:22.649508 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649493    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5psr\" (UniqueName: \"kubernetes.io/projected/2470a645-deb4-4505-8937-5b5bb75df8d2-kube-api-access-h5psr\") pod \"2470a645-deb4-4505-8937-5b5bb75df8d2\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") "
Jul 24 17:38:22.649745 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649523    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls\") pod \"2470a645-deb4-4505-8937-5b5bb75df8d2\" (UID: \"2470a645-deb4-4505-8937-5b5bb75df8d2\") "
Jul 24 17:38:22.649745 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649680    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2470a645-deb4-4505-8937-5b5bb75df8d2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2470a645-deb4-4505-8937-5b5bb75df8d2" (UID: "2470a645-deb4-4505-8937-5b5bb75df8d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:38:22.649854 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649782    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d62711be-165d-4942-96d0-0c3d54f1eae2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d62711be-165d-4942-96d0-0c3d54f1eae2" (UID: "d62711be-165d-4942-96d0-0c3d54f1eae2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:38:22.649854 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649798    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2470a645-deb4-4505-8937-5b5bb75df8d2-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:22.649854 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.649824    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c803fdc-7c4a-4332-a6f2-128251ffa074-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c803fdc-7c4a-4332-a6f2-128251ffa074" (UID: "0c803fdc-7c4a-4332-a6f2-128251ffa074"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:38:22.651783 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.651755    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2470a645-deb4-4505-8937-5b5bb75df8d2-kube-api-access-h5psr" (OuterVolumeSpecName: "kube-api-access-h5psr") pod "2470a645-deb4-4505-8937-5b5bb75df8d2" (UID: "2470a645-deb4-4505-8937-5b5bb75df8d2"). InnerVolumeSpecName "kube-api-access-h5psr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:38:22.651920 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.651832    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c803fdc-7c4a-4332-a6f2-128251ffa074-kube-api-access-5t49x" (OuterVolumeSpecName: "kube-api-access-5t49x") pod "0c803fdc-7c4a-4332-a6f2-128251ffa074" (UID: "0c803fdc-7c4a-4332-a6f2-128251ffa074"). InnerVolumeSpecName "kube-api-access-5t49x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:38:22.651920 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.651851    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2470a645-deb4-4505-8937-5b5bb75df8d2" (UID: "2470a645-deb4-4505-8937-5b5bb75df8d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:38:22.651920 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.651861    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d62711be-165d-4942-96d0-0c3d54f1eae2-kube-api-access-xv29f" (OuterVolumeSpecName: "kube-api-access-xv29f") pod "d62711be-165d-4942-96d0-0c3d54f1eae2" (UID: "d62711be-165d-4942-96d0-0c3d54f1eae2"). InnerVolumeSpecName "kube-api-access-xv29f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:38:22.750496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.750457    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv29f\" (UniqueName: \"kubernetes.io/projected/d62711be-165d-4942-96d0-0c3d54f1eae2-kube-api-access-xv29f\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:22.750496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.750490    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5psr\" (UniqueName: \"kubernetes.io/projected/2470a645-deb4-4505-8937-5b5bb75df8d2-kube-api-access-h5psr\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:22.750689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.750504    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2470a645-deb4-4505-8937-5b5bb75df8d2-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:22.750689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.750516    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d62711be-165d-4942-96d0-0c3d54f1eae2-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:22.750689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.750528    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t49x\" (UniqueName: \"kubernetes.io/projected/0c803fdc-7c4a-4332-a6f2-128251ffa074-kube-api-access-5t49x\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:22.750689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:22.750539    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c803fdc-7c4a-4332-a6f2-128251ffa074-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:23.446590 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.446560    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"
Jul 24 17:38:23.447050 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.446564    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc" event={"ID":"0c803fdc-7c4a-4332-a6f2-128251ffa074","Type":"ContainerDied","Data":"1834d48f78b125c60284f6ca96c95b841f4ab1872b97488f211f4a2f4cf45d7c"}
Jul 24 17:38:23.447050 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.446716    2562 scope.go:117] "RemoveContainer" containerID="710c6eba2b9908114708d49bb658e0afd48be5c06509b67cd9894c01e2eddb5f"
Jul 24 17:38:23.448038 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.447998    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"
Jul 24 17:38:23.448115 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.448036    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v" event={"ID":"d62711be-165d-4942-96d0-0c3d54f1eae2","Type":"ContainerDied","Data":"bdcacc9187805aaa316feca2c04508bda9bf1ca294dc56dbfdf1bafb70245f98"}
Jul 24 17:38:23.449393 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.449374    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj" event={"ID":"2470a645-deb4-4505-8937-5b5bb75df8d2","Type":"ContainerDied","Data":"6f75b04c3c298dab612f94aaf080e24edfe9867c1e634472e7ae9071f1e8ed52"}
Jul 24 17:38:23.449474 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.449409    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"
Jul 24 17:38:23.456107 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.456091    2562 scope.go:117] "RemoveContainer" containerID="2a7d5d951e610f2f09fdc57fdad10848197f617b4376e9cffd80e1ced82c6203"
Jul 24 17:38:23.464968 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.464948    2562 scope.go:117] "RemoveContainer" containerID="36b6007a5a854e3823d9e74ece49e80c952a794e12289616e03e0653df71af46"
Jul 24 17:38:23.474774 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.474710    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"]
Jul 24 17:38:23.476864 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.476843    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-f431f60c-5467dd5d44-dmh9v"]
Jul 24 17:38:23.488211 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.488187    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"]
Jul 24 17:38:23.492960 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.492929    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-f431f60c-bfcdfc7df-6jwxc"]
Jul 24 17:38:23.505761 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.505734    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c803fdc-7c4a-4332-a6f2-128251ffa074" path="/var/lib/kubelet/pods/0c803fdc-7c4a-4332-a6f2-128251ffa074/volumes"
Jul 24 17:38:23.506103 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.506091    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d62711be-165d-4942-96d0-0c3d54f1eae2" path="/var/lib/kubelet/pods/d62711be-165d-4942-96d0-0c3d54f1eae2/volumes"
Jul 24 17:38:23.506331 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.506319    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"]
Jul 24 17:38:23.510462 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:23.510438    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f431f60c-558f45f98f-s2jsj"]
Jul 24 17:38:25.505232 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:25.505196    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2470a645-deb4-4505-8937-5b5bb75df8d2" path="/var/lib/kubelet/pods/2470a645-deb4-4505-8937-5b5bb75df8d2/volumes"
Jul 24 17:38:27.015090 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.015050    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"]
Jul 24 17:38:27.015510 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.015329    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy" containerID="cri-o://810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9" gracePeriod=30
Jul 24 17:38:27.804418 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804376    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"]
Jul 24 17:38:27.804829 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804815    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy"
Jul 24 17:38:27.804876 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804834    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy"
Jul 24 17:38:27.804876 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804845    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c803fdc-7c4a-4332-a6f2-128251ffa074" containerName="payload-processing"
Jul 24 17:38:27.804876 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804853    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c803fdc-7c4a-4332-a6f2-128251ffa074" containerName="payload-processing"
Jul 24 17:38:27.804876 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804873    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d62711be-165d-4942-96d0-0c3d54f1eae2" containerName="payload-pre-processing"
Jul 24 17:38:27.805001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804881    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d62711be-165d-4942-96d0-0c3d54f1eae2" containerName="payload-pre-processing"
Jul 24 17:38:27.805001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804899    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2470a645-deb4-4505-8937-5b5bb75df8d2" containerName="maas-api"
Jul 24 17:38:27.805001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804907    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2470a645-deb4-4505-8937-5b5bb75df8d2" containerName="maas-api"
Jul 24 17:38:27.805001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804971    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy"
Jul 24 17:38:27.805001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804983    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy"
Jul 24 17:38:27.805001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.804994    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d62711be-165d-4942-96d0-0c3d54f1eae2" containerName="payload-pre-processing"
Jul 24 17:38:27.805268 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.805002    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c803fdc-7c4a-4332-a6f2-128251ffa074" containerName="payload-processing"
Jul 24 17:38:27.805268 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.805012    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2470a645-deb4-4505-8937-5b5bb75df8d2" containerName="maas-api"
Jul 24 17:38:27.809808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.809784    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.812506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.812484    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e9e1f4e1-openshift-default-dockercfg-k7dx2\""
Jul 24 17:38:27.822415 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.822388    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"]
Jul 24 17:38:27.892446 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-certs\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.892673 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-podinfo\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.892745 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-envoy\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.892802 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-socket\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.892855 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892838    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-credential-socket\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.892918 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd6dd\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-kube-api-access-cd6dd\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.892918 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892902    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istiod-ca-cert\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.893011 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-token\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.893011 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.892983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-data\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.993899 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.993859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-envoy\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.993899 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.993903    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-socket\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.993954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-credential-socket\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.993985    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd6dd\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-kube-api-access-cd6dd\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994009    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istiod-ca-cert\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994078    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-token\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-data\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-certs\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994183    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-podinfo\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994516 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994381    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-socket\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994640 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-credential-socket\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994727 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994689    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-certs\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.994776 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.994747    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-data\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.995079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.995061    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istiod-ca-cert\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.997045 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.996993    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-envoy\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:27.997327 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:27.997305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-podinfo\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:28.002899 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.002874    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-token\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:28.003049 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.003008    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd6dd\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-kube-api-access-cd6dd\") pod \"e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") " pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:28.120183 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.120095    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:28.259350 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.259317    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"]
Jul 24 17:38:28.261875 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:38:28.261839    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbbfd5c8_23df_418c_9e0b_10396d3b5889.slice/crio-3d57535a5c3719f805546748e63f1ccc2db41c850bf3e78fead3e1867d98c29f WatchSource:0}: Error finding container 3d57535a5c3719f805546748e63f1ccc2db41c850bf3e78fead3e1867d98c29f: Status 404 returned error can't find the container with id 3d57535a5c3719f805546748e63f1ccc2db41c850bf3e78fead3e1867d98c29f
Jul 24 17:38:28.264173 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.264137    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:28.264272 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.264216    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:28.264272 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.264266    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:28.475424 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.475390    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" event={"ID":"dbbfd5c8-23df-418c-9e0b-10396d3b5889","Type":"ContainerStarted","Data":"f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476"}
Jul 24 17:38:28.475608 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.475450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" event={"ID":"dbbfd5c8-23df-418c-9e0b-10396d3b5889","Type":"ContainerStarted","Data":"3d57535a5c3719f805546748e63f1ccc2db41c850bf3e78fead3e1867d98c29f"}
Jul 24 17:38:28.513573 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:28.513506    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" podStartSLOduration=1.5134830419999998 podStartE2EDuration="1.513483042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:38:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:38:28.512460788 +0000 UTC m=+1697.483592981" watchObservedRunningTime="2026-07-24 17:38:28.513483042 +0000 UTC m=+1697.484615224"
Jul 24 17:38:29.120817 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:29.120783    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:29.126370 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:29.126343    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:29.480791 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:29.480761    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:29.481905 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:29.481883    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:30.115411 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.115378    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"]
Jul 24 17:38:30.115691 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.115678    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy"
Jul 24 17:38:30.115760 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.115691    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78289f81-c7a1-46f2-b070-65d05acd3fd4" containerName="istio-proxy"
Jul 24 17:38:30.118694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.118667    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.123880 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.123855    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e9e1f4e1-serving-cert\""
Jul 24 17:38:30.123880 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.123857    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:38:30.125202 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.125175    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:38:30.125304 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.125274    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:38:30.131134 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.131109    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"]
Jul 24 17:38:30.216548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.216502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.216737 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.216603    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kc8d\" (UniqueName: \"kubernetes.io/projected/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-kube-api-access-2kc8d\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.216737 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.216645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-openshift-service-ca\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.317797 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.317753    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.317797 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.317800    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kc8d\" (UniqueName: \"kubernetes.io/projected/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-kube-api-access-2kc8d\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.318076 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:38:30.317894    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-serving-cert: secret "maas-api-e2e-ait-e9e1f4e1-serving-cert" not found
Jul 24 17:38:30.318076 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.317928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-openshift-service-ca\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.318076 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:38:30.317961    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls podName:6b8746e3-5146-4b2f-84b9-4e9d71df2ff6 nodeName:}" failed. No retries permitted until 2026-07-24 17:38:30.817944277 +0000 UTC m=+1699.789076436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls") pod "maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442" (UID: "6b8746e3-5146-4b2f-84b9-4e9d71df2ff6") : secret "maas-api-e2e-ait-e9e1f4e1-serving-cert" not found
Jul 24 17:38:30.318607 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.318588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-openshift-service-ca\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.327959 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.327936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kc8d\" (UniqueName: \"kubernetes.io/projected/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-kube-api-access-2kc8d\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.772511 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.772476    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"]
Jul 24 17:38:30.776151 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.776126    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:30.778922 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.778897    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e9e1f4e1\""
Jul 24 17:38:30.779056 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.778947    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e9e1f4e1-dockercfg-zjsqk\""
Jul 24 17:38:30.788114 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.788088    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"]
Jul 24 17:38:30.823295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.823259    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.825987 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.825963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls\") pod \"maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:30.924358 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.924320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b80b497-9053-425f-90b6-b0ab1a1b2167-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w\" (UID: \"4b80b497-9053-425f-90b6-b0ab1a1b2167\") " pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:30.924553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:30.924383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs2dc\" (UniqueName: \"kubernetes.io/projected/4b80b497-9053-425f-90b6-b0ab1a1b2167-kube-api-access-bs2dc\") pod \"payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w\" (UID: \"4b80b497-9053-425f-90b6-b0ab1a1b2167\") " pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:31.025453 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.025367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b80b497-9053-425f-90b6-b0ab1a1b2167-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w\" (UID: \"4b80b497-9053-425f-90b6-b0ab1a1b2167\") " pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:31.025453 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.025418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs2dc\" (UniqueName: \"kubernetes.io/projected/4b80b497-9053-425f-90b6-b0ab1a1b2167-kube-api-access-bs2dc\") pod \"payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w\" (UID: \"4b80b497-9053-425f-90b6-b0ab1a1b2167\") " pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:31.026136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.026111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b80b497-9053-425f-90b6-b0ab1a1b2167-plugins-config-volume\") pod \"payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w\" (UID: \"4b80b497-9053-425f-90b6-b0ab1a1b2167\") " pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:31.030305 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.030283    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:31.034076 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.034054    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs2dc\" (UniqueName: \"kubernetes.io/projected/4b80b497-9053-425f-90b6-b0ab1a1b2167-kube-api-access-bs2dc\") pod \"payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w\" (UID: \"4b80b497-9053-425f-90b6-b0ab1a1b2167\") " pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:31.088132 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.087758    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:31.175267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.175237    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"]
Jul 24 17:38:31.176529 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:38:31.176501    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8746e3_5146_4b2f_84b9_4e9d71df2ff6.slice/crio-e4ec658716a6f96a2ab5155ffacf5725b8a831010202a0c6b392c9f05ed74bdc WatchSource:0}: Error finding container e4ec658716a6f96a2ab5155ffacf5725b8a831010202a0c6b392c9f05ed74bdc: Status 404 returned error can't find the container with id e4ec658716a6f96a2ab5155ffacf5725b8a831010202a0c6b392c9f05ed74bdc
Jul 24 17:38:31.241922 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.241897    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"]
Jul 24 17:38:31.244398 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:38:31.244372    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b80b497_9053_425f_90b6_b0ab1a1b2167.slice/crio-2c78e27901fe58b5223249b31ebce19d022f5ab65eafdf79a0ea640c58e634ae WatchSource:0}: Error finding container 2c78e27901fe58b5223249b31ebce19d022f5ab65eafdf79a0ea640c58e634ae: Status 404 returned error can't find the container with id 2c78e27901fe58b5223249b31ebce19d022f5ab65eafdf79a0ea640c58e634ae
Jul 24 17:38:31.490620 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.490584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442" event={"ID":"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6","Type":"ContainerStarted","Data":"e4ec658716a6f96a2ab5155ffacf5725b8a831010202a0c6b392c9f05ed74bdc"}
Jul 24 17:38:31.492334 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.492307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w" event={"ID":"4b80b497-9053-425f-90b6-b0ab1a1b2167","Type":"ContainerStarted","Data":"2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae"}
Jul 24 17:38:31.492509 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.492354    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w" event={"ID":"4b80b497-9053-425f-90b6-b0ab1a1b2167","Type":"ContainerStarted","Data":"2c78e27901fe58b5223249b31ebce19d022f5ab65eafdf79a0ea640c58e634ae"}
Jul 24 17:38:31.509937 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:31.509891    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w" podStartSLOduration=1.509874669 podStartE2EDuration="1.509874669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:38:31.508187038 +0000 UTC m=+1700.479319220" watchObservedRunningTime="2026-07-24 17:38:31.509874669 +0000 UTC m=+1700.481006849"
Jul 24 17:38:32.275760 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.275734    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:38:32.439469 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439370    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-podinfo\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.439469 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439463    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-token\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.439707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439504    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-certs\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.439707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439535    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-data\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.439707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439567    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-socket\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.439707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439593    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-envoy\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.439707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439637    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfzh9\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-kube-api-access-gfzh9\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.439707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439674    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istiod-ca-cert\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.439707 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439702    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-credential-socket\") pod \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\" (UID: \"56a1042c-64e6-4eec-aa91-54d1e67b8e6c\") "
Jul 24 17:38:32.440128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439776    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:38:32.440128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.439897    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:38:32.440128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.440073    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:32.440278 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.440169    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:32.440535 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.440402    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:38:32.440535 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.440488    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:38:32.440719 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.440683    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-data" (OuterVolumeSpecName: "istio-data") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:38:32.441916 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.441872    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:38:32.441916 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.441901    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-kube-api-access-gfzh9" (OuterVolumeSpecName: "kube-api-access-gfzh9") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "kube-api-access-gfzh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:38:32.442073 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.441955    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:38:32.442073 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.441959    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-token" (OuterVolumeSpecName: "istio-token") pod "56a1042c-64e6-4eec-aa91-54d1e67b8e6c" (UID: "56a1042c-64e6-4eec-aa91-54d1e67b8e6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:38:32.498584 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.498552    2562 generic.go:356] "Generic (PLEG): container finished" podID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerID="810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9" exitCode=0
Jul 24 17:38:32.498747 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.498605    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"
Jul 24 17:38:32.498747 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.498646    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" event={"ID":"56a1042c-64e6-4eec-aa91-54d1e67b8e6c","Type":"ContainerDied","Data":"810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9"}
Jul 24 17:38:32.498747 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.498715    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr" event={"ID":"56a1042c-64e6-4eec-aa91-54d1e67b8e6c","Type":"ContainerDied","Data":"8f7cecb706c550455528c8ac401e93c6e6337eea0cd8dcd5d9d2b2c772433ff3"}
Jul 24 17:38:32.498900 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.498749    2562 scope.go:117] "RemoveContainer" containerID="810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9"
Jul 24 17:38:32.500273 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.500248    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442" event={"ID":"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6","Type":"ContainerStarted","Data":"b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed"}
Jul 24 17:38:32.500621 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.500523    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:32.500621 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.500573    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:32.507791 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.507775    2562 scope.go:117] "RemoveContainer" containerID="4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152"
Jul 24 17:38:32.515521 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.515502    2562 scope.go:117] "RemoveContainer" containerID="810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9"
Jul 24 17:38:32.515785 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:38:32.515768    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9\": container with ID starting with 810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9 not found: ID does not exist" containerID="810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9"
Jul 24 17:38:32.515849 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.515794    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9"} err="failed to get container status \"810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9\": rpc error: code = NotFound desc = could not find container \"810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9\": container with ID starting with 810268a1efd0f9527ec030e46fa9505ae87cc4ce3da59df7238a25323d487dd9 not found: ID does not exist"
Jul 24 17:38:32.515849 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.515812    2562 scope.go:117] "RemoveContainer" containerID="4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152"
Jul 24 17:38:32.516081 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:38:32.516059    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152\": container with ID starting with 4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152 not found: ID does not exist" containerID="4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152"
Jul 24 17:38:32.516167 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.516090    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152"} err="failed to get container status \"4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152\": rpc error: code = NotFound desc = could not find container \"4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152\": container with ID starting with 4ae074b1bad6b51e5a9d0d9fa534550f67ed5c63aa7dd9bf8970d82e1f71b152 not found: ID does not exist"
Jul 24 17:38:32.521610 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.521553    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442" podStartSLOduration=2.115315142 podStartE2EDuration="2.521536557s" totalImagesPullingTime="406.221415ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:38:32.519375917 +0000 UTC m=+1701.490508098" watchObservedRunningTime="2026-07-24 17:38:32.521536557 +0000 UTC m=+1701.492668739"
Jul 24 17:38:32.532637 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.532611    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"]
Jul 24 17:38:32.535930 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.535903    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f431f60c-gw-openshift-default-5bc8fbf644-zr4cr"]
Jul 24 17:38:32.540571 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.540551    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:32.540639 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.540573    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:32.540639 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.540583    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:32.540639 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.540592    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:32.540639 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.540599    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:32.540639 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.540606    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:32.540639 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:32.540613    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfzh9\" (UniqueName: \"kubernetes.io/projected/56a1042c-64e6-4eec-aa91-54d1e67b8e6c-kube-api-access-gfzh9\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:38:33.505448 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:33.505407    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" path="/var/lib/kubelet/pods/56a1042c-64e6-4eec-aa91-54d1e67b8e6c/volumes"
Jul 24 17:38:38.511737 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:38.511708    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:38:43.508793 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:43.508759    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:38:44.607724 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:44.607645    2562 patch_prober.go:28] interesting pod/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": read tcp 10.133.0.2:53700->10.133.0.36:15021: read: connection reset by peer" start-of-body=
Jul 24 17:38:44.608131 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:44.607716    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": read tcp 10.133.0.2:53700->10.133.0.36:15021: read: connection reset by peer"
Jul 24 17:38:45.559718 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:45.559684    2562 generic.go:356] "Generic (PLEG): container finished" podID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerID="f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476" exitCode=0
Jul 24 17:38:45.559894 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:45.559756    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" event={"ID":"dbbfd5c8-23df-418c-9e0b-10396d3b5889","Type":"ContainerDied","Data":"f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476"}
Jul 24 17:38:45.560156 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:45.560144    2562 scope.go:117] "RemoveContainer" containerID="f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476"
Jul 24 17:38:45.561129 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:45.561101    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:45.561189 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:45.561167    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:45.561228 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:45.561198    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:38:46.568972 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:46.568924    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" event={"ID":"dbbfd5c8-23df-418c-9e0b-10396d3b5889","Type":"ContainerStarted","Data":"55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c"}
Jul 24 17:38:47.120470 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:47.120432    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:47.120722 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:47.120697    2562 patch_prober.go:28] interesting pod/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 24 17:38:47.120806 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:47.120753    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 24 17:38:48.121551 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:48.121513    2562 patch_prober.go:28] interesting pod/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 24 17:38:48.121927 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:48.121572    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 24 17:38:49.121175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:49.121141    2562 patch_prober.go:28] interesting pod/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 24 17:38:49.121345 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:49.121194    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 24 17:38:50.120687 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:50.120652    2562 patch_prober.go:28] interesting pod/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body=
Jul 24 17:38:50.121097 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:50.120706    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused"
Jul 24 17:38:51.124138 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:51.124109    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:51.124589 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:51.124435    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:38:51.125197 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:38:51.125183    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:39:06.141082 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.141047    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"]
Jul 24 17:39:06.141634 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.141253    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442" podUID="6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" containerName="maas-api" containerID="cri-o://b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed" gracePeriod=30
Jul 24 17:39:06.188654 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.188622    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"]
Jul 24 17:39:06.188899 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.188855    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w" podUID="4b80b497-9053-425f-90b6-b0ab1a1b2167" containerName="payload-processing" containerID="cri-o://2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae" gracePeriod=30
Jul 24 17:39:06.436689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.436662    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:39:06.464947 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.464924    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:39:06.552407 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.552379    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-openshift-service-ca\") pod \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") "
Jul 24 17:39:06.552584 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.552420    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kc8d\" (UniqueName: \"kubernetes.io/projected/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-kube-api-access-2kc8d\") pod \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") "
Jul 24 17:39:06.552584 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.552441    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls\") pod \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\" (UID: \"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6\") "
Jul 24 17:39:06.552584 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.552456    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs2dc\" (UniqueName: \"kubernetes.io/projected/4b80b497-9053-425f-90b6-b0ab1a1b2167-kube-api-access-bs2dc\") pod \"4b80b497-9053-425f-90b6-b0ab1a1b2167\" (UID: \"4b80b497-9053-425f-90b6-b0ab1a1b2167\") "
Jul 24 17:39:06.552584 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.552533    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b80b497-9053-425f-90b6-b0ab1a1b2167-plugins-config-volume\") pod \"4b80b497-9053-425f-90b6-b0ab1a1b2167\" (UID: \"4b80b497-9053-425f-90b6-b0ab1a1b2167\") "
Jul 24 17:39:06.552900 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.552871    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" (UID: "6b8746e3-5146-4b2f-84b9-4e9d71df2ff6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:39:06.553054 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.552990    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b80b497-9053-425f-90b6-b0ab1a1b2167-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b80b497-9053-425f-90b6-b0ab1a1b2167" (UID: "4b80b497-9053-425f-90b6-b0ab1a1b2167"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:39:06.554683 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.554660    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" (UID: "6b8746e3-5146-4b2f-84b9-4e9d71df2ff6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:39:06.554767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.554685    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-kube-api-access-2kc8d" (OuterVolumeSpecName: "kube-api-access-2kc8d") pod "6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" (UID: "6b8746e3-5146-4b2f-84b9-4e9d71df2ff6"). InnerVolumeSpecName "kube-api-access-2kc8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:39:06.554767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.554717    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b80b497-9053-425f-90b6-b0ab1a1b2167-kube-api-access-bs2dc" (OuterVolumeSpecName: "kube-api-access-bs2dc") pod "4b80b497-9053-425f-90b6-b0ab1a1b2167" (UID: "4b80b497-9053-425f-90b6-b0ab1a1b2167"). InnerVolumeSpecName "kube-api-access-bs2dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:39:06.653882 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.653806    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b80b497-9053-425f-90b6-b0ab1a1b2167-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:06.653882 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.653834    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:06.653882 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.653843    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kc8d\" (UniqueName: \"kubernetes.io/projected/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-kube-api-access-2kc8d\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:06.653882 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.653852    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:06.653882 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.653861    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs2dc\" (UniqueName: \"kubernetes.io/projected/4b80b497-9053-425f-90b6-b0ab1a1b2167-kube-api-access-bs2dc\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:06.658778 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.658753    2562 generic.go:356] "Generic (PLEG): container finished" podID="6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" containerID="b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed" exitCode=0
Jul 24 17:39:06.658915 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.658797    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"
Jul 24 17:39:06.658915 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.658830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442" event={"ID":"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6","Type":"ContainerDied","Data":"b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed"}
Jul 24 17:39:06.658915 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.658868    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442" event={"ID":"6b8746e3-5146-4b2f-84b9-4e9d71df2ff6","Type":"ContainerDied","Data":"e4ec658716a6f96a2ab5155ffacf5725b8a831010202a0c6b392c9f05ed74bdc"}
Jul 24 17:39:06.658915 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.658893    2562 scope.go:117] "RemoveContainer" containerID="b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed"
Jul 24 17:39:06.660121 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.660092    2562 generic.go:356] "Generic (PLEG): container finished" podID="4b80b497-9053-425f-90b6-b0ab1a1b2167" containerID="2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae" exitCode=0
Jul 24 17:39:06.660237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.660130    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"
Jul 24 17:39:06.660237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.660169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w" event={"ID":"4b80b497-9053-425f-90b6-b0ab1a1b2167","Type":"ContainerDied","Data":"2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae"}
Jul 24 17:39:06.660237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.660206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w" event={"ID":"4b80b497-9053-425f-90b6-b0ab1a1b2167","Type":"ContainerDied","Data":"2c78e27901fe58b5223249b31ebce19d022f5ab65eafdf79a0ea640c58e634ae"}
Jul 24 17:39:06.668160 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.668143    2562 scope.go:117] "RemoveContainer" containerID="b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed"
Jul 24 17:39:06.668432 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:39:06.668414    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed\": container with ID starting with b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed not found: ID does not exist" containerID="b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed"
Jul 24 17:39:06.668512 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.668444    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed"} err="failed to get container status \"b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed\": rpc error: code = NotFound desc = could not find container \"b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed\": container with ID starting with b892ab02f868111d1fe068389a79436bf121edd9dbfa41a67ffb5047f18634ed not found: ID does not exist"
Jul 24 17:39:06.668512 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.668466    2562 scope.go:117] "RemoveContainer" containerID="2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae"
Jul 24 17:39:06.675763 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.675747    2562 scope.go:117] "RemoveContainer" containerID="2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae"
Jul 24 17:39:06.676005 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:39:06.675987    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae\": container with ID starting with 2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae not found: ID does not exist" containerID="2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae"
Jul 24 17:39:06.676139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.676011    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae"} err="failed to get container status \"2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae\": rpc error: code = NotFound desc = could not find container \"2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae\": container with ID starting with 2961d8dd2a73611af109afd7fe148a610ea3a46e2f70af59ca1aba5a68b6a3ae not found: ID does not exist"
Jul 24 17:39:06.679816 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.679791    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"]
Jul 24 17:39:06.682262 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.682242    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e9e1f4e1-54f9d9446c-hz442"]
Jul 24 17:39:06.691920 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.691899    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"]
Jul 24 17:39:06.695606 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:06.695588    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e9e1f4e1-5db47c88dd-9hs8w"]
Jul 24 17:39:07.504752 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:07.504719    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b80b497-9053-425f-90b6-b0ab1a1b2167" path="/var/lib/kubelet/pods/4b80b497-9053-425f-90b6-b0ab1a1b2167/volumes"
Jul 24 17:39:07.505293 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:07.505096    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" path="/var/lib/kubelet/pods/6b8746e3-5146-4b2f-84b9-4e9d71df2ff6/volumes"
Jul 24 17:39:11.857841 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:11.857764    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"]
Jul 24 17:39:11.858230 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:11.857970    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy" containerID="cri-o://55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c" gracePeriod=30
Jul 24 17:39:12.624418 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624382    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"]
Jul 24 17:39:12.624839 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624825    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy"
Jul 24 17:39:12.624898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624843    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy"
Jul 24 17:39:12.624898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624854    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" containerName="maas-api"
Jul 24 17:39:12.624898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624864    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" containerName="maas-api"
Jul 24 17:39:12.624999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624901    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b80b497-9053-425f-90b6-b0ab1a1b2167" containerName="payload-processing"
Jul 24 17:39:12.624999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624910    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b80b497-9053-425f-90b6-b0ab1a1b2167" containerName="payload-processing"
Jul 24 17:39:12.624999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624973    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy"
Jul 24 17:39:12.624999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624984    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b8746e3-5146-4b2f-84b9-4e9d71df2ff6" containerName="maas-api"
Jul 24 17:39:12.624999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.624994    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b80b497-9053-425f-90b6-b0ab1a1b2167" containerName="payload-processing"
Jul 24 17:39:12.625221 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.625003    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy"
Jul 24 17:39:12.628216 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.628199    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.630953 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.630932    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3efc9f0f-openshift-default-dockercfg-hln9l\""
Jul 24 17:39:12.639486 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.639464    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"]
Jul 24 17:39:12.708788 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.708744    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93f315e-a6da-4de6-9dff-c472faec335c-istio-podinfo\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.709110 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.709085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93f315e-a6da-4de6-9dff-c472faec335c-istiod-ca-cert\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.709335 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.709314    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-credential-socket\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.709434 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.709356    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-envoy\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.709434 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.709399    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-istio-token\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.709544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.709450    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-data\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.709544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.709485    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-certs\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.709544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.709527    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-socket\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.709694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.709555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk6ck\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-kube-api-access-pk6ck\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.810795 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.810755    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-data\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.810984 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.810806    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-certs\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.810984 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.810842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-socket\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.810984 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.810872    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk6ck\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-kube-api-access-pk6ck\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.810984 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.810935    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93f315e-a6da-4de6-9dff-c472faec335c-istio-podinfo\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.810984 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.810961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93f315e-a6da-4de6-9dff-c472faec335c-istiod-ca-cert\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.811260 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.811001    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-credential-socket\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.811260 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.811149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-envoy\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.811260 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.811192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-istio-token\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.811260 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.811211    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-data\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.811469 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.811300    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-socket\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.811469 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.811329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-credential-socket\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.811569 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.811498    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-certs\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.811726 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.811700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93f315e-a6da-4de6-9dff-c472faec335c-istiod-ca-cert\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.813520 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.813492    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-envoy\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.813706 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.813689    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93f315e-a6da-4de6-9dff-c472faec335c-istio-podinfo\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.819488 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.819468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-istio-token\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.819667 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.819650    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk6ck\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-kube-api-access-pk6ck\") pod \"e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") " pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:12.939203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:12.939113    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:13.080370 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:13.080345    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"]
Jul 24 17:39:13.082215 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:39:13.082188    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93f315e_a6da_4de6_9dff_c472faec335c.slice/crio-1f0366f4ef67514f1036c51c1b17a600ccf69842967f89203b3ebf03e79899fe WatchSource:0}: Error finding container 1f0366f4ef67514f1036c51c1b17a600ccf69842967f89203b3ebf03e79899fe: Status 404 returned error can't find the container with id 1f0366f4ef67514f1036c51c1b17a600ccf69842967f89203b3ebf03e79899fe
Jul 24 17:39:13.084412 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:13.084369    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:39:13.084464 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:13.084444    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:39:13.084506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:13.084473    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:39:13.692491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:13.692458    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" event={"ID":"a93f315e-a6da-4de6-9dff-c472faec335c","Type":"ContainerStarted","Data":"f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c"}
Jul 24 17:39:13.692691 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:13.692511    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" event={"ID":"a93f315e-a6da-4de6-9dff-c472faec335c","Type":"ContainerStarted","Data":"1f0366f4ef67514f1036c51c1b17a600ccf69842967f89203b3ebf03e79899fe"}
Jul 24 17:39:13.715423 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:13.715370    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" podStartSLOduration=1.715357075 podStartE2EDuration="1.715357075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:39:13.714075159 +0000 UTC m=+1742.685207337" watchObservedRunningTime="2026-07-24 17:39:13.715357075 +0000 UTC m=+1742.686489254"
Jul 24 17:39:13.939493 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:13.939459    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:14.944597 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:14.944561    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:15.118445 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.118413    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"]
Jul 24 17:39:15.118728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.118714    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy"
Jul 24 17:39:15.118801 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.118729    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="56a1042c-64e6-4eec-aa91-54d1e67b8e6c" containerName="istio-proxy"
Jul 24 17:39:15.121801 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.121778    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.124541 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.124518    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:39:15.124673 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.124542    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:39:15.124791 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.124777    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-3efc9f0f-serving-cert\""
Jul 24 17:39:15.124927 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.124916    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:39:15.133181 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.133156    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"]
Jul 24 17:39:15.233151 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.233117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.233374 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.233187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgbnt\" (UniqueName: \"kubernetes.io/projected/76ad41a6-f2fa-47e1-a391-58e0647ab608-kube-api-access-dgbnt\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.233374 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.233274    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ad41a6-f2fa-47e1-a391-58e0647ab608-openshift-service-ca\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.333723 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.333691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ad41a6-f2fa-47e1-a391-58e0647ab608-openshift-service-ca\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.333920 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.333739    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.333920 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.333786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgbnt\" (UniqueName: \"kubernetes.io/projected/76ad41a6-f2fa-47e1-a391-58e0647ab608-kube-api-access-dgbnt\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.333920 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:39:15.333912    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-serving-cert: secret "maas-api-e2e-derive-3efc9f0f-serving-cert" not found
Jul 24 17:39:15.334152 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:39:15.333980    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls podName:76ad41a6-f2fa-47e1-a391-58e0647ab608 nodeName:}" failed. No retries permitted until 2026-07-24 17:39:15.833962098 +0000 UTC m=+1744.805094261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls") pod "maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4" (UID: "76ad41a6-f2fa-47e1-a391-58e0647ab608") : secret "maas-api-e2e-derive-3efc9f0f-serving-cert" not found
Jul 24 17:39:15.334404 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.334386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ad41a6-f2fa-47e1-a391-58e0647ab608-openshift-service-ca\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.345726 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.345703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgbnt\" (UniqueName: \"kubernetes.io/projected/76ad41a6-f2fa-47e1-a391-58e0647ab608-kube-api-access-dgbnt\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.700527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.700501    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:15.701467 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.701447    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:15.746164 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.746137    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"]
Jul 24 17:39:15.749723 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.749701    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:15.752547 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.752525    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-3efc9f0f-dockercfg-87jx8\""
Jul 24 17:39:15.752693 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.752603    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-3efc9f0f\""
Jul 24 17:39:15.758986 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.758518    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"]
Jul 24 17:39:15.839264 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.839228    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd5ll\" (UniqueName: \"kubernetes.io/projected/85321725-9c86-45c5-90d1-0491c0416857-kube-api-access-jd5ll\") pod \"payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l\" (UID: \"85321725-9c86-45c5-90d1-0491c0416857\") " pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:15.839445 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.839392    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.839538 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.839496    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85321725-9c86-45c5-90d1-0491c0416857-plugins-config-volume\") pod \"payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l\" (UID: \"85321725-9c86-45c5-90d1-0491c0416857\") " pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:15.841834 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.841813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls\") pod \"maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:15.940100 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.940069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85321725-9c86-45c5-90d1-0491c0416857-plugins-config-volume\") pod \"payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l\" (UID: \"85321725-9c86-45c5-90d1-0491c0416857\") " pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:15.940275 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.940118    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd5ll\" (UniqueName: \"kubernetes.io/projected/85321725-9c86-45c5-90d1-0491c0416857-kube-api-access-jd5ll\") pod \"payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l\" (UID: \"85321725-9c86-45c5-90d1-0491c0416857\") " pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:15.940768 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.940748    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85321725-9c86-45c5-90d1-0491c0416857-plugins-config-volume\") pod \"payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l\" (UID: \"85321725-9c86-45c5-90d1-0491c0416857\") " pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:15.948824 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:15.948803    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd5ll\" (UniqueName: \"kubernetes.io/projected/85321725-9c86-45c5-90d1-0491c0416857-kube-api-access-jd5ll\") pod \"payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l\" (UID: \"85321725-9c86-45c5-90d1-0491c0416857\") " pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:16.034654 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.034573    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:16.063491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.063464    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:16.186927 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.186891    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"]
Jul 24 17:39:16.191123 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:39:16.191092    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ad41a6_f2fa_47e1_a391_58e0647ab608.slice/crio-e8c9bdda77cbe780e5c576b3faad2d4d58cca96ef544d4cf7fe9b1a7cd8c1065 WatchSource:0}: Error finding container e8c9bdda77cbe780e5c576b3faad2d4d58cca96ef544d4cf7fe9b1a7cd8c1065: Status 404 returned error can't find the container with id e8c9bdda77cbe780e5c576b3faad2d4d58cca96ef544d4cf7fe9b1a7cd8c1065
Jul 24 17:39:16.214657 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.214634    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"]
Jul 24 17:39:16.216930 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:39:16.216903    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85321725_9c86_45c5_90d1_0491c0416857.slice/crio-26c7835dbd59d360d16ce8f91e0de758b18c5fa5b4c6d0e6088ab2757c73d0e7 WatchSource:0}: Error finding container 26c7835dbd59d360d16ce8f91e0de758b18c5fa5b4c6d0e6088ab2757c73d0e7: Status 404 returned error can't find the container with id 26c7835dbd59d360d16ce8f91e0de758b18c5fa5b4c6d0e6088ab2757c73d0e7
Jul 24 17:39:16.706255 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.706218    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l" event={"ID":"85321725-9c86-45c5-90d1-0491c0416857","Type":"ContainerStarted","Data":"b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0"}
Jul 24 17:39:16.706455 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.706276    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l" event={"ID":"85321725-9c86-45c5-90d1-0491c0416857","Type":"ContainerStarted","Data":"26c7835dbd59d360d16ce8f91e0de758b18c5fa5b4c6d0e6088ab2757c73d0e7"}
Jul 24 17:39:16.706455 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.706307    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:16.707708 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.707682    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4" event={"ID":"76ad41a6-f2fa-47e1-a391-58e0647ab608","Type":"ContainerStarted","Data":"8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27"}
Jul 24 17:39:16.707817 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.707725    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4" event={"ID":"76ad41a6-f2fa-47e1-a391-58e0647ab608","Type":"ContainerStarted","Data":"e8c9bdda77cbe780e5c576b3faad2d4d58cca96ef544d4cf7fe9b1a7cd8c1065"}
Jul 24 17:39:16.707893 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.707875    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:16.721989 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.721945    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l" podStartSLOduration=1.721925712 podStartE2EDuration="1.721925712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:39:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:39:16.721723118 +0000 UTC m=+1745.692855297" watchObservedRunningTime="2026-07-24 17:39:16.721925712 +0000 UTC m=+1745.693057870"
Jul 24 17:39:16.739334 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:16.739286    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4" podStartSLOduration=1.336920831 podStartE2EDuration="1.739269558s" totalImagesPullingTime="402.348727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:39:16.737545833 +0000 UTC m=+1745.708678011" watchObservedRunningTime="2026-07-24 17:39:16.739269558 +0000 UTC m=+1745.710401778"
Jul 24 17:39:17.098099 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.098076    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:39:17.252928 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.252894    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-data\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.252942    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istiod-ca-cert\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.252968    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-credential-socket\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.252993    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-token\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253054    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-envoy\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253094    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-podinfo\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253139    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-socket\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253164    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd6dd\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-kube-api-access-cd6dd\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253195    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-certs\") pod \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\" (UID: \"dbbfd5c8-23df-418c-9e0b-10396d3b5889\") "
Jul 24 17:39:17.253458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253288    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:39:17.253458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253405    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:39:17.253643 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253516    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:39:17.253643 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253520    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.253643 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253564    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.253643 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253621    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:39:17.254042 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.253978    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-data" (OuterVolumeSpecName: "istio-data") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:39:17.255499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.255480    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-token" (OuterVolumeSpecName: "istio-token") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:39:17.255685 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.255661    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:39:17.255962 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.255937    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:39:17.255962 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.255940    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-kube-api-access-cd6dd" (OuterVolumeSpecName: "kube-api-access-cd6dd") pod "dbbfd5c8-23df-418c-9e0b-10396d3b5889" (UID: "dbbfd5c8-23df-418c-9e0b-10396d3b5889"). InnerVolumeSpecName "kube-api-access-cd6dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:39:17.354418 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.354389    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.354418 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.354417    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.354578 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.354428    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.354578 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.354439    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbbfd5c8-23df-418c-9e0b-10396d3b5889-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.354578 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.354450    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.354578 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.354461    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd6dd\" (UniqueName: \"kubernetes.io/projected/dbbfd5c8-23df-418c-9e0b-10396d3b5889-kube-api-access-cd6dd\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.354578 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.354475    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbbfd5c8-23df-418c-9e0b-10396d3b5889-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:17.713238 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.713201    2562 generic.go:356] "Generic (PLEG): container finished" podID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerID="55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c" exitCode=0
Jul 24 17:39:17.713416 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.713248    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"
Jul 24 17:39:17.713416 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.713297    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" event={"ID":"dbbfd5c8-23df-418c-9e0b-10396d3b5889","Type":"ContainerDied","Data":"55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c"}
Jul 24 17:39:17.713416 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.713350    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t" event={"ID":"dbbfd5c8-23df-418c-9e0b-10396d3b5889","Type":"ContainerDied","Data":"3d57535a5c3719f805546748e63f1ccc2db41c850bf3e78fead3e1867d98c29f"}
Jul 24 17:39:17.713416 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.713372    2562 scope.go:117] "RemoveContainer" containerID="55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c"
Jul 24 17:39:17.721970 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.721957    2562 scope.go:117] "RemoveContainer" containerID="f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476"
Jul 24 17:39:17.729361 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.729337    2562 scope.go:117] "RemoveContainer" containerID="55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c"
Jul 24 17:39:17.729869 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:39:17.729740    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c\": container with ID starting with 55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c not found: ID does not exist" containerID="55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c"
Jul 24 17:39:17.729869 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.729783    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c"} err="failed to get container status \"55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c\": rpc error: code = NotFound desc = could not find container \"55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c\": container with ID starting with 55f679a813d35f1ee459a759bc9c06db7f3e1236d547e083e6810f459b2ca25c not found: ID does not exist"
Jul 24 17:39:17.729869 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.729805    2562 scope.go:117] "RemoveContainer" containerID="f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476"
Jul 24 17:39:17.730192 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:39:17.730095    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476\": container with ID starting with f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476 not found: ID does not exist" containerID="f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476"
Jul 24 17:39:17.730192 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.730124    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476"} err="failed to get container status \"f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476\": rpc error: code = NotFound desc = could not find container \"f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476\": container with ID starting with f190aa37fbc8d1d8dd82a2931b9f05aa86510bf76a0a9056f2246a3befbdd476 not found: ID does not exist"
Jul 24 17:39:17.731872 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.731850    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"]
Jul 24 17:39:17.738012 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:17.737991    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e9e1f4e1-openshift-default-74798c94cb-8vp9t"]
Jul 24 17:39:19.504786 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:19.504751    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" path="/var/lib/kubelet/pods/dbbfd5c8-23df-418c-9e0b-10396d3b5889/volumes"
Jul 24 17:39:22.719625 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:22.719594    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:27.715677 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:27.715642    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:28.766143 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:28.766108    2562 generic.go:356] "Generic (PLEG): container finished" podID="a93f315e-a6da-4de6-9dff-c472faec335c" containerID="f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c" exitCode=0
Jul 24 17:39:28.766522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:28.766151    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" event={"ID":"a93f315e-a6da-4de6-9dff-c472faec335c","Type":"ContainerDied","Data":"f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c"}
Jul 24 17:39:28.766568 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:28.766527    2562 scope.go:117] "RemoveContainer" containerID="f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c"
Jul 24 17:39:28.767523 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:28.767494    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:39:28.767595 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:28.767579    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:39:28.767640 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:28.767625    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:39:29.775259 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:29.775228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" event={"ID":"a93f315e-a6da-4de6-9dff-c472faec335c","Type":"ContainerStarted","Data":"f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b"}
Jul 24 17:39:29.939626 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:29.939596    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:29.940647 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:29.940616    2562 patch_prober.go:28] interesting pod/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 24 17:39:29.940723 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:29.940660    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 24 17:39:30.700881 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:30.700853    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:30.940208 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:30.940172    2562 patch_prober.go:28] interesting pod/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 24 17:39:30.940694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:30.940234    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 24 17:39:31.940237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:31.940199    2562 patch_prober.go:28] interesting pod/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 24 17:39:31.940747 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:31.940264    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 24 17:39:32.940277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:32.940242    2562 patch_prober.go:28] interesting pod/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Jul 24 17:39:32.940658 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:32.940301    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Jul 24 17:39:33.943341 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:33.943309    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:33.944445 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:33.944426    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:39:50.291162 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.291128    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"]
Jul 24 17:39:50.291650 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.291330    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4" podUID="76ad41a6-f2fa-47e1-a391-58e0647ab608" containerName="maas-api" containerID="cri-o://8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27" gracePeriod=30
Jul 24 17:39:50.336832 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.336802    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"]
Jul 24 17:39:50.337099 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.337058    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l" podUID="85321725-9c86-45c5-90d1-0491c0416857" containerName="payload-processing" containerID="cri-o://b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0" gracePeriod=30
Jul 24 17:39:50.585423 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.585397    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:50.616147 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.616118    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:50.637840 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.637812    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls\") pod \"76ad41a6-f2fa-47e1-a391-58e0647ab608\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") "
Jul 24 17:39:50.637983 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.637861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85321725-9c86-45c5-90d1-0491c0416857-plugins-config-volume\") pod \"85321725-9c86-45c5-90d1-0491c0416857\" (UID: \"85321725-9c86-45c5-90d1-0491c0416857\") "
Jul 24 17:39:50.637983 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.637907    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgbnt\" (UniqueName: \"kubernetes.io/projected/76ad41a6-f2fa-47e1-a391-58e0647ab608-kube-api-access-dgbnt\") pod \"76ad41a6-f2fa-47e1-a391-58e0647ab608\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") "
Jul 24 17:39:50.638125 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.637983    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ad41a6-f2fa-47e1-a391-58e0647ab608-openshift-service-ca\") pod \"76ad41a6-f2fa-47e1-a391-58e0647ab608\" (UID: \"76ad41a6-f2fa-47e1-a391-58e0647ab608\") "
Jul 24 17:39:50.638125 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.638044    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd5ll\" (UniqueName: \"kubernetes.io/projected/85321725-9c86-45c5-90d1-0491c0416857-kube-api-access-jd5ll\") pod \"85321725-9c86-45c5-90d1-0491c0416857\" (UID: \"85321725-9c86-45c5-90d1-0491c0416857\") "
Jul 24 17:39:50.638339 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.638301    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85321725-9c86-45c5-90d1-0491c0416857-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85321725-9c86-45c5-90d1-0491c0416857" (UID: "85321725-9c86-45c5-90d1-0491c0416857"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:39:50.638577 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.638553    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76ad41a6-f2fa-47e1-a391-58e0647ab608-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "76ad41a6-f2fa-47e1-a391-58e0647ab608" (UID: "76ad41a6-f2fa-47e1-a391-58e0647ab608"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:39:50.640431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.640404    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85321725-9c86-45c5-90d1-0491c0416857-kube-api-access-jd5ll" (OuterVolumeSpecName: "kube-api-access-jd5ll") pod "85321725-9c86-45c5-90d1-0491c0416857" (UID: "85321725-9c86-45c5-90d1-0491c0416857"). InnerVolumeSpecName "kube-api-access-jd5ll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:39:50.640524 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.640445    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "76ad41a6-f2fa-47e1-a391-58e0647ab608" (UID: "76ad41a6-f2fa-47e1-a391-58e0647ab608"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:39:50.640524 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.640460    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76ad41a6-f2fa-47e1-a391-58e0647ab608-kube-api-access-dgbnt" (OuterVolumeSpecName: "kube-api-access-dgbnt") pod "76ad41a6-f2fa-47e1-a391-58e0647ab608" (UID: "76ad41a6-f2fa-47e1-a391-58e0647ab608"). InnerVolumeSpecName "kube-api-access-dgbnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:39:50.739384 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.739355    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76ad41a6-f2fa-47e1-a391-58e0647ab608-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:50.739384 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.739382    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85321725-9c86-45c5-90d1-0491c0416857-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:50.739574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.739394    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgbnt\" (UniqueName: \"kubernetes.io/projected/76ad41a6-f2fa-47e1-a391-58e0647ab608-kube-api-access-dgbnt\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:50.739574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.739406    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ad41a6-f2fa-47e1-a391-58e0647ab608-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:50.739574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.739418    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd5ll\" (UniqueName: \"kubernetes.io/projected/85321725-9c86-45c5-90d1-0491c0416857-kube-api-access-jd5ll\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:39:50.863606 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.863517    2562 generic.go:356] "Generic (PLEG): container finished" podID="85321725-9c86-45c5-90d1-0491c0416857" containerID="b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0" exitCode=0
Jul 24 17:39:50.863606 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.863560    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"
Jul 24 17:39:50.863806 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.863611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l" event={"ID":"85321725-9c86-45c5-90d1-0491c0416857","Type":"ContainerDied","Data":"b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0"}
Jul 24 17:39:50.863806 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.863672    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l" event={"ID":"85321725-9c86-45c5-90d1-0491c0416857","Type":"ContainerDied","Data":"26c7835dbd59d360d16ce8f91e0de758b18c5fa5b4c6d0e6088ab2757c73d0e7"}
Jul 24 17:39:50.863806 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.863705    2562 scope.go:117] "RemoveContainer" containerID="b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0"
Jul 24 17:39:50.864892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.864870    2562 generic.go:356] "Generic (PLEG): container finished" podID="76ad41a6-f2fa-47e1-a391-58e0647ab608" containerID="8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27" exitCode=0
Jul 24 17:39:50.864962 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.864906    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"
Jul 24 17:39:50.864962 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.864929    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4" event={"ID":"76ad41a6-f2fa-47e1-a391-58e0647ab608","Type":"ContainerDied","Data":"8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27"}
Jul 24 17:39:50.865066 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.864971    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4" event={"ID":"76ad41a6-f2fa-47e1-a391-58e0647ab608","Type":"ContainerDied","Data":"e8c9bdda77cbe780e5c576b3faad2d4d58cca96ef544d4cf7fe9b1a7cd8c1065"}
Jul 24 17:39:50.872143 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.872100    2562 scope.go:117] "RemoveContainer" containerID="b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0"
Jul 24 17:39:50.872977 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:39:50.872955    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0\": container with ID starting with b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0 not found: ID does not exist" containerID="b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0"
Jul 24 17:39:50.873092 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.872989    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0"} err="failed to get container status \"b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0\": rpc error: code = NotFound desc = could not find container \"b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0\": container with ID starting with b720b93bf5620fbb0e570387c32385057fde4c8f3ac1d48e133c22e156b7e3c0 not found: ID does not exist"
Jul 24 17:39:50.873092 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.873012    2562 scope.go:117] "RemoveContainer" containerID="8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27"
Jul 24 17:39:50.881197 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.881168    2562 scope.go:117] "RemoveContainer" containerID="8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27"
Jul 24 17:39:50.881433 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:39:50.881416    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27\": container with ID starting with 8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27 not found: ID does not exist" containerID="8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27"
Jul 24 17:39:50.881476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.881439    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27"} err="failed to get container status \"8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27\": rpc error: code = NotFound desc = could not find container \"8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27\": container with ID starting with 8bedcd3926c3a10d6b0c676f1e61e8b7f17ea8f4a83c279018ead73976061f27 not found: ID does not exist"
Jul 24 17:39:50.888426 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.888404    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"]
Jul 24 17:39:50.891813 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.891793    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-3efc9f0f-6c7fc55f58-xcrw4"]
Jul 24 17:39:50.901840 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.901817    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"]
Jul 24 17:39:50.905830 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:50.905807    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3efc9f0f-5b8589fb74-thm9l"]
Jul 24 17:39:51.504792 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:51.504763    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76ad41a6-f2fa-47e1-a391-58e0647ab608" path="/var/lib/kubelet/pods/76ad41a6-f2fa-47e1-a391-58e0647ab608/volumes"
Jul 24 17:39:51.505154 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:51.505102    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85321725-9c86-45c5-90d1-0491c0416857" path="/var/lib/kubelet/pods/85321725-9c86-45c5-90d1-0491c0416857/volumes"
Jul 24 17:39:55.168392 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:55.168360    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"]
Jul 24 17:39:55.168915 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:39:55.168546    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy" containerID="cri-o://f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b" gracePeriod=30
Jul 24 17:40:00.422722 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.422699    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:40:00.519541 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519505    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-envoy\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519554    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-credential-socket\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519590    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-istio-token\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519609    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-certs\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519634    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-data\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519699    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk6ck\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-kube-api-access-pk6ck\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519737    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93f315e-a6da-4de6-9dff-c472faec335c-istiod-ca-cert\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519763    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93f315e-a6da-4de6-9dff-c472faec335c-istio-podinfo\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519797    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-socket\") pod \"a93f315e-a6da-4de6-9dff-c472faec335c\" (UID: \"a93f315e-a6da-4de6-9dff-c472faec335c\") "
Jul 24 17:40:00.519996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519909    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:40:00.519996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.519925    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:40:00.520277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.520096    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.520277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.520116    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.520277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.520174    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:40:00.520466 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.520440    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a93f315e-a6da-4de6-9dff-c472faec335c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:40:00.520641 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.520605    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-data" (OuterVolumeSpecName: "istio-data") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:40:00.521892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.521864    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-istio-token" (OuterVolumeSpecName: "istio-token") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:40:00.521892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.521874    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:40:00.522070 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.521987    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a93f315e-a6da-4de6-9dff-c472faec335c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:40:00.522282 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.522264    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-kube-api-access-pk6ck" (OuterVolumeSpecName: "kube-api-access-pk6ck") pod "a93f315e-a6da-4de6-9dff-c472faec335c" (UID: "a93f315e-a6da-4de6-9dff-c472faec335c"). InnerVolumeSpecName "kube-api-access-pk6ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:40:00.621294 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.621250    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.621294 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.621289    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.621294 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.621299    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk6ck\" (UniqueName: \"kubernetes.io/projected/a93f315e-a6da-4de6-9dff-c472faec335c-kube-api-access-pk6ck\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.621294 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.621309    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a93f315e-a6da-4de6-9dff-c472faec335c-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.621555 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.621317    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a93f315e-a6da-4de6-9dff-c472faec335c-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.621555 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.621325    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.621555 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.621332    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a93f315e-a6da-4de6-9dff-c472faec335c-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:40:00.908307 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.908213    2562 generic.go:356] "Generic (PLEG): container finished" podID="a93f315e-a6da-4de6-9dff-c472faec335c" containerID="f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b" exitCode=0
Jul 24 17:40:00.908307 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.908289    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"
Jul 24 17:40:00.908527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.908303    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" event={"ID":"a93f315e-a6da-4de6-9dff-c472faec335c","Type":"ContainerDied","Data":"f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b"}
Jul 24 17:40:00.908527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.908393    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h" event={"ID":"a93f315e-a6da-4de6-9dff-c472faec335c","Type":"ContainerDied","Data":"1f0366f4ef67514f1036c51c1b17a600ccf69842967f89203b3ebf03e79899fe"}
Jul 24 17:40:00.908527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.908419    2562 scope.go:117] "RemoveContainer" containerID="f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b"
Jul 24 17:40:00.917671 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.917656    2562 scope.go:117] "RemoveContainer" containerID="f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c"
Jul 24 17:40:00.925081 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.925065    2562 scope.go:117] "RemoveContainer" containerID="f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b"
Jul 24 17:40:00.925321 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:40:00.925305    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b\": container with ID starting with f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b not found: ID does not exist" containerID="f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b"
Jul 24 17:40:00.925373 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.925329    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b"} err="failed to get container status \"f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b\": rpc error: code = NotFound desc = could not find container \"f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b\": container with ID starting with f0b27fca0dfe46937cae5cacc111b80d2392713bbe9f27553d3ce9377cd2534b not found: ID does not exist"
Jul 24 17:40:00.925373 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.925345    2562 scope.go:117] "RemoveContainer" containerID="f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c"
Jul 24 17:40:00.925563 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:40:00.925546    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c\": container with ID starting with f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c not found: ID does not exist" containerID="f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c"
Jul 24 17:40:00.925598 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.925569    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c"} err="failed to get container status \"f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c\": rpc error: code = NotFound desc = could not find container \"f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c\": container with ID starting with f4759c891204be3249b9cf0d9a422f838d9e4d478b6ffaf65aeaef66ff40be7c not found: ID does not exist"
Jul 24 17:40:00.929188 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.929165    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"]
Jul 24 17:40:00.935406 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:00.935386    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3efc9f0f-openshift-default-68958f8dd4-wbr9h"]
Jul 24 17:40:01.506986 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:01.506944    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" path="/var/lib/kubelet/pods/a93f315e-a6da-4de6-9dff-c472faec335c/volumes"
Jul 24 17:40:11.531690 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:11.531659    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:40:11.534084 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:40:11.534062    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:41:37.759835 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.759753    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"]
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760187    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760206    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760218    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760227    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760239    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760247    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760261    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76ad41a6-f2fa-47e1-a391-58e0647ab608" containerName="maas-api"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760269    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="76ad41a6-f2fa-47e1-a391-58e0647ab608" containerName="maas-api"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760279    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760287    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760306    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85321725-9c86-45c5-90d1-0491c0416857" containerName="payload-processing"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760314    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="85321725-9c86-45c5-90d1-0491c0416857" containerName="payload-processing"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760395    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy"
Jul 24 17:41:37.760403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760405    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy"
Jul 24 17:41:37.761103 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760417    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="76ad41a6-f2fa-47e1-a391-58e0647ab608" containerName="maas-api"
Jul 24 17:41:37.761103 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.760427    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="85321725-9c86-45c5-90d1-0491c0416857" containerName="payload-processing"
Jul 24 17:41:37.763510 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.763491    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.766278 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.766259    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-cccec6a6-openshift-default-dockercfg-29dt2\""
Jul 24 17:41:37.784986 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.784962    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"]
Jul 24 17:41:37.862345 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-podinfo\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.862544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862359    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xd76\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-kube-api-access-6xd76\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.862544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-socket\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.862544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862438    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-credential-socket\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.862544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-token\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.862544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862514    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-data\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.862805 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-envoy\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.862805 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-certs\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.862805 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.862625    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istiod-ca-cert\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.963679 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.963646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-envoy\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.963679 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.963681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-certs\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.963926 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.963702    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istiod-ca-cert\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.963926 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.963843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-podinfo\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.963926 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.963878    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xd76\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-kube-api-access-6xd76\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.963926 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.963902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-socket\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.964179 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.963930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-credential-socket\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.964179 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.963980    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-token\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.964179 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.964043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-data\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.964179 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.964118    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-certs\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.964406 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.964381    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-credential-socket\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.964546 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.964411    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-socket\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.964546 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.964491    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istiod-ca-cert\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.964546 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.964570    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-data\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.966381 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.966357    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-envoy\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.966822 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.966797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-podinfo\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.979495 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.979470    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-token\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:37.979635 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:37.979508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xd76\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-kube-api-access-6xd76\") pod \"e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") " pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:38.074214 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:38.074121    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:38.219323 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:38.219278    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"]
Jul 24 17:41:38.224122 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:41:38.224088    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad21d6d_6127_4ea3_b8e2_0f7155354f8c.slice/crio-dc21e910ae4a9da71532fae07329921d7bf2771f82c617d357dfc08cc34ebe6b WatchSource:0}: Error finding container dc21e910ae4a9da71532fae07329921d7bf2771f82c617d357dfc08cc34ebe6b: Status 404 returned error can't find the container with id dc21e910ae4a9da71532fae07329921d7bf2771f82c617d357dfc08cc34ebe6b
Jul 24 17:41:38.226171 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:38.226135    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:41:38.226268 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:38.226248    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:41:38.226332 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:38.226296    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:41:38.321365 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:38.321327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" event={"ID":"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c","Type":"ContainerStarted","Data":"d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8"}
Jul 24 17:41:38.321563 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:38.321392    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" event={"ID":"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c","Type":"ContainerStarted","Data":"dc21e910ae4a9da71532fae07329921d7bf2771f82c617d357dfc08cc34ebe6b"}
Jul 24 17:41:38.345543 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:38.345434    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" podStartSLOduration=1.345414419 podStartE2EDuration="1.345414419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:38.342815106 +0000 UTC m=+1887.313947287" watchObservedRunningTime="2026-07-24 17:41:38.345414419 +0000 UTC m=+1887.316546601"
Jul 24 17:41:39.074771 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:39.074734    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:39.079557 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:39.079535    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:39.326262 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:39.326181    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:39.326976 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:39.326958    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:40.713049 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.712996    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"]
Jul 24 17:41:40.713579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.713542    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbbfd5c8-23df-418c-9e0b-10396d3b5889" containerName="istio-proxy"
Jul 24 17:41:40.713665 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.713585    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a93f315e-a6da-4de6-9dff-c472faec335c" containerName="istio-proxy"
Jul 24 17:41:40.716543 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.716508    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:40.719798 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.719775    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:41:40.719958 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.719858    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-cccec6a6-serving-cert\""
Jul 24 17:41:40.719958 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.719933    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:41:40.721098 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.721079    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:41:40.727546 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.727508    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"]
Jul 24 17:41:40.793043 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.792977    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4vvp\" (UniqueName: \"kubernetes.io/projected/b8550012-8faa-4f06-9095-cd4aca5cb22e-kube-api-access-p4vvp\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:40.793263 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.793051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8550012-8faa-4f06-9095-cd4aca5cb22e-openshift-service-ca\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:40.793263 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.793180    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:40.893826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.893786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:40.894087 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.893845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4vvp\" (UniqueName: \"kubernetes.io/projected/b8550012-8faa-4f06-9095-cd4aca5cb22e-kube-api-access-p4vvp\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:40.894087 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:41:40.893954    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-serving-cert: secret "maas-api-e2e-mt-cccec6a6-serving-cert" not found
Jul 24 17:41:40.894087 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.893960    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8550012-8faa-4f06-9095-cd4aca5cb22e-openshift-service-ca\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:40.894087 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:41:40.894069    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls podName:b8550012-8faa-4f06-9095-cd4aca5cb22e nodeName:}" failed. No retries permitted until 2026-07-24 17:41:41.394046447 +0000 UTC m=+1890.365178623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls") pod "maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg" (UID: "b8550012-8faa-4f06-9095-cd4aca5cb22e") : secret "maas-api-e2e-mt-cccec6a6-serving-cert" not found
Jul 24 17:41:40.894561 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.894544    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8550012-8faa-4f06-9095-cd4aca5cb22e-openshift-service-ca\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:40.905165 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:40.905141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4vvp\" (UniqueName: \"kubernetes.io/projected/b8550012-8faa-4f06-9095-cd4aca5cb22e-kube-api-access-p4vvp\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:41.378349 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.378313    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"]
Jul 24 17:41:41.381497 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.381480    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:41.384710 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.384683    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-cccec6a6\""
Jul 24 17:41:41.393081 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.393051    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"]
Jul 24 17:41:41.398633 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.398603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:41.401432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.401408    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls\") pod \"maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:41.424753 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.424715    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"]
Jul 24 17:41:41.427942 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.427919    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:41.431245 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.431225    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-cccec6a6-dockercfg-q2gc7\""
Jul 24 17:41:41.446415 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.446386    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"]
Jul 24 17:41:41.499233 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.499193    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ggk4\" (UniqueName: \"kubernetes.io/projected/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-kube-api-access-5ggk4\") pod \"payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd\" (UID: \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\") " pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:41.499430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.499259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/839ba90d-0bdc-4d94-9b1e-82987093c7cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95\" (UID: \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:41.499430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.499328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-plugins-config-volume\") pod \"payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd\" (UID: \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\") " pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:41.499430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.499352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkqlh\" (UniqueName: \"kubernetes.io/projected/839ba90d-0bdc-4d94-9b1e-82987093c7cc-kube-api-access-lkqlh\") pod \"payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95\" (UID: \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:41.600405 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.600370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/839ba90d-0bdc-4d94-9b1e-82987093c7cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95\" (UID: \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:41.600635 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.600425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-plugins-config-volume\") pod \"payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd\" (UID: \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\") " pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:41.600635 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.600460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkqlh\" (UniqueName: \"kubernetes.io/projected/839ba90d-0bdc-4d94-9b1e-82987093c7cc-kube-api-access-lkqlh\") pod \"payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95\" (UID: \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:41.600635 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.600509    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ggk4\" (UniqueName: \"kubernetes.io/projected/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-kube-api-access-5ggk4\") pod \"payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd\" (UID: \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\") " pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:41.601186 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.601166    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/839ba90d-0bdc-4d94-9b1e-82987093c7cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95\" (UID: \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:41.601303 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.601286    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-plugins-config-volume\") pod \"payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd\" (UID: \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\") " pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:41.610809 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.610781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkqlh\" (UniqueName: \"kubernetes.io/projected/839ba90d-0bdc-4d94-9b1e-82987093c7cc-kube-api-access-lkqlh\") pod \"payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95\" (UID: \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:41.610971 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.610901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ggk4\" (UniqueName: \"kubernetes.io/projected/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-kube-api-access-5ggk4\") pod \"payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd\" (UID: \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\") " pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:41.628877 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.628782    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:41.692562 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.692521    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:41.738447 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.738385    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:41.780363 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.780297    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"]
Jul 24 17:41:41.784284 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:41:41.784254    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8550012_8faa_4f06_9095_cd4aca5cb22e.slice/crio-ce1dec1885a1594fc773a95f9d2b6f3164ae96045e0c34c4099465965ada7b16 WatchSource:0}: Error finding container ce1dec1885a1594fc773a95f9d2b6f3164ae96045e0c34c4099465965ada7b16: Status 404 returned error can't find the container with id ce1dec1885a1594fc773a95f9d2b6f3164ae96045e0c34c4099465965ada7b16
Jul 24 17:41:41.858198 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.858154    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"]
Jul 24 17:41:41.860361 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:41:41.860335    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839ba90d_0bdc_4d94_9b1e_82987093c7cc.slice/crio-a360e75c05c0ecc1efe1f79aba92250229956caab6529f6a30ffddd07c39f364 WatchSource:0}: Error finding container a360e75c05c0ecc1efe1f79aba92250229956caab6529f6a30ffddd07c39f364: Status 404 returned error can't find the container with id a360e75c05c0ecc1efe1f79aba92250229956caab6529f6a30ffddd07c39f364
Jul 24 17:41:41.894356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:41.894332    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"]
Jul 24 17:41:41.900354 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:41:41.900320    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c8eeb2_4f06_4e4b_a3e6_3dc6e39ab11a.slice/crio-27ace0906576649a007c69995e1c12f6bab82eae5fc353910537df9fb1120aac WatchSource:0}: Error finding container 27ace0906576649a007c69995e1c12f6bab82eae5fc353910537df9fb1120aac: Status 404 returned error can't find the container with id 27ace0906576649a007c69995e1c12f6bab82eae5fc353910537df9fb1120aac
Jul 24 17:41:42.339255 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.339220    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd" event={"ID":"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a","Type":"ContainerStarted","Data":"4ab8f695e50f1c552014d59acb3b9efd5491a5f9b88490c8b024eac095354f8f"}
Jul 24 17:41:42.339396 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.339279    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd" event={"ID":"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a","Type":"ContainerStarted","Data":"27ace0906576649a007c69995e1c12f6bab82eae5fc353910537df9fb1120aac"}
Jul 24 17:41:42.339396 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.339320    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:42.340753 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.340727    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg" event={"ID":"b8550012-8faa-4f06-9095-cd4aca5cb22e","Type":"ContainerStarted","Data":"8b7fc3745ae5d85af637bbd8c4a84c63eb94dc1661332af9f004ac1e68be567c"}
Jul 24 17:41:42.340871 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.340772    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:42.340871 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.340797    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg" event={"ID":"b8550012-8faa-4f06-9095-cd4aca5cb22e","Type":"ContainerStarted","Data":"ce1dec1885a1594fc773a95f9d2b6f3164ae96045e0c34c4099465965ada7b16"}
Jul 24 17:41:42.342139 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.342118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95" event={"ID":"839ba90d-0bdc-4d94-9b1e-82987093c7cc","Type":"ContainerStarted","Data":"f7360b7571ec22713a10bfafbf67de9e1be30a754e8616daac65bc38e4b9df06"}
Jul 24 17:41:42.342222 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.342153    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95" event={"ID":"839ba90d-0bdc-4d94-9b1e-82987093c7cc","Type":"ContainerStarted","Data":"a360e75c05c0ecc1efe1f79aba92250229956caab6529f6a30ffddd07c39f364"}
Jul 24 17:41:42.342260 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.342240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:42.360928 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.360870    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd" podStartSLOduration=1.36085032 podStartE2EDuration="1.36085032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:42.357382959 +0000 UTC m=+1891.328515144" watchObservedRunningTime="2026-07-24 17:41:42.36085032 +0000 UTC m=+1891.331982502"
Jul 24 17:41:42.373888 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.373834    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95" podStartSLOduration=1.373816871 podStartE2EDuration="1.373816871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:42.373186602 +0000 UTC m=+1891.344318784" watchObservedRunningTime="2026-07-24 17:41:42.373816871 +0000 UTC m=+1891.344949050"
Jul 24 17:41:42.391839 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:42.391792    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg" podStartSLOduration=1.942387683 podStartE2EDuration="2.391774543s" totalImagesPullingTime="449.38686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:42.389713651 +0000 UTC m=+1891.360845832" watchObservedRunningTime="2026-07-24 17:41:42.391774543 +0000 UTC m=+1891.362906724"
Jul 24 17:41:48.363948 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:48.363914    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:41:52.396773 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:52.396733    2562 generic.go:356] "Generic (PLEG): container finished" podID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerID="d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8" exitCode=0
Jul 24 17:41:52.397226 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:52.396813    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" event={"ID":"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c","Type":"ContainerDied","Data":"d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8"}
Jul 24 17:41:52.397226 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:52.397219    2562 scope.go:117] "RemoveContainer" containerID="d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8"
Jul 24 17:41:52.398360 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:52.398332    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:41:52.398433 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:52.398416    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:41:52.398493 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:52.398476    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:41:53.351942 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:53.351901    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:41:53.353211 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:53.353182    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:41:53.404179 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:53.404003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" event={"ID":"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c","Type":"ContainerStarted","Data":"68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992"}
Jul 24 17:41:54.075252 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:54.075207    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:54.076588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:54.076561    2562 patch_prober.go:28] interesting pod/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 24 17:41:54.076694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:54.076628    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 24 17:41:54.326992 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:54.326907    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:55.075534 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:55.075497    2562 patch_prober.go:28] interesting pod/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 24 17:41:55.075927 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:55.075569    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 24 17:41:56.074745 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:56.074706    2562 patch_prober.go:28] interesting pod/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 24 17:41:56.074940 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:56.074765    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 24 17:41:57.074499 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:57.074466    2562 patch_prober.go:28] interesting pod/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 24 17:41:57.074908 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:57.074527    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 24 17:41:58.079278 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:58.079248    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:41:58.080185 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:41:58.080166    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:42:02.248551 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.248504    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"]
Jul 24 17:42:02.248978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.248729    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg" podUID="b8550012-8faa-4f06-9095-cd4aca5cb22e" containerName="maas-api" containerID="cri-o://8b7fc3745ae5d85af637bbd8c4a84c63eb94dc1661332af9f004ac1e68be567c" gracePeriod=30
Jul 24 17:42:02.301447 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.301413    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"]
Jul 24 17:42:02.301724 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.301670    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd" podUID="a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a" containerName="payload-processing" containerID="cri-o://4ab8f695e50f1c552014d59acb3b9efd5491a5f9b88490c8b024eac095354f8f" gracePeriod=30
Jul 24 17:42:02.309146 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.309117    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"]
Jul 24 17:42:02.309385 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.309357    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95" podUID="839ba90d-0bdc-4d94-9b1e-82987093c7cc" containerName="payload-pre-processing" containerID="cri-o://f7360b7571ec22713a10bfafbf67de9e1be30a754e8616daac65bc38e4b9df06" gracePeriod=30
Jul 24 17:42:02.444703 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.444665    2562 generic.go:356] "Generic (PLEG): container finished" podID="839ba90d-0bdc-4d94-9b1e-82987093c7cc" containerID="f7360b7571ec22713a10bfafbf67de9e1be30a754e8616daac65bc38e4b9df06" exitCode=0
Jul 24 17:42:02.444907 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.444740    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95" event={"ID":"839ba90d-0bdc-4d94-9b1e-82987093c7cc","Type":"ContainerDied","Data":"f7360b7571ec22713a10bfafbf67de9e1be30a754e8616daac65bc38e4b9df06"}
Jul 24 17:42:02.448533 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.448499    2562 generic.go:356] "Generic (PLEG): container finished" podID="a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a" containerID="4ab8f695e50f1c552014d59acb3b9efd5491a5f9b88490c8b024eac095354f8f" exitCode=0
Jul 24 17:42:02.448708 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.448532    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd" event={"ID":"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a","Type":"ContainerDied","Data":"4ab8f695e50f1c552014d59acb3b9efd5491a5f9b88490c8b024eac095354f8f"}
Jul 24 17:42:02.450456 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.450425    2562 generic.go:356] "Generic (PLEG): container finished" podID="b8550012-8faa-4f06-9095-cd4aca5cb22e" containerID="8b7fc3745ae5d85af637bbd8c4a84c63eb94dc1661332af9f004ac1e68be567c" exitCode=0
Jul 24 17:42:02.450547 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.450472    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg" event={"ID":"b8550012-8faa-4f06-9095-cd4aca5cb22e","Type":"ContainerDied","Data":"8b7fc3745ae5d85af637bbd8c4a84c63eb94dc1661332af9f004ac1e68be567c"}
Jul 24 17:42:02.562461 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.562433    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:42:02.597985 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.597955    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:42:02.605054 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.605033    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:42:02.691038 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.690987    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-plugins-config-volume\") pod \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\" (UID: \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\") "
Jul 24 17:42:02.691205 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.691107    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ggk4\" (UniqueName: \"kubernetes.io/projected/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-kube-api-access-5ggk4\") pod \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\" (UID: \"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a\") "
Jul 24 17:42:02.691403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.691370    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a" (UID: "a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:02.693281 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.693259    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-kube-api-access-5ggk4" (OuterVolumeSpecName: "kube-api-access-5ggk4") pod "a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a" (UID: "a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a"). InnerVolumeSpecName "kube-api-access-5ggk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:02.791648 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.791567    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls\") pod \"b8550012-8faa-4f06-9095-cd4aca5cb22e\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") "
Jul 24 17:42:02.791648 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.791606    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkqlh\" (UniqueName: \"kubernetes.io/projected/839ba90d-0bdc-4d94-9b1e-82987093c7cc-kube-api-access-lkqlh\") pod \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\" (UID: \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\") "
Jul 24 17:42:02.791648 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.791638    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4vvp\" (UniqueName: \"kubernetes.io/projected/b8550012-8faa-4f06-9095-cd4aca5cb22e-kube-api-access-p4vvp\") pod \"b8550012-8faa-4f06-9095-cd4aca5cb22e\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") "
Jul 24 17:42:02.791837 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.791758    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/839ba90d-0bdc-4d94-9b1e-82987093c7cc-plugins-config-volume\") pod \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\" (UID: \"839ba90d-0bdc-4d94-9b1e-82987093c7cc\") "
Jul 24 17:42:02.791837 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.791801    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8550012-8faa-4f06-9095-cd4aca5cb22e-openshift-service-ca\") pod \"b8550012-8faa-4f06-9095-cd4aca5cb22e\" (UID: \"b8550012-8faa-4f06-9095-cd4aca5cb22e\") "
Jul 24 17:42:02.792128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.792106    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ggk4\" (UniqueName: \"kubernetes.io/projected/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-kube-api-access-5ggk4\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:02.792267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.792131    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:02.792267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.792179    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/839ba90d-0bdc-4d94-9b1e-82987093c7cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "839ba90d-0bdc-4d94-9b1e-82987093c7cc" (UID: "839ba90d-0bdc-4d94-9b1e-82987093c7cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:02.792267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.792197    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8550012-8faa-4f06-9095-cd4aca5cb22e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8550012-8faa-4f06-9095-cd4aca5cb22e" (UID: "b8550012-8faa-4f06-9095-cd4aca5cb22e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:02.793898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.793870    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/839ba90d-0bdc-4d94-9b1e-82987093c7cc-kube-api-access-lkqlh" (OuterVolumeSpecName: "kube-api-access-lkqlh") pod "839ba90d-0bdc-4d94-9b1e-82987093c7cc" (UID: "839ba90d-0bdc-4d94-9b1e-82987093c7cc"). InnerVolumeSpecName "kube-api-access-lkqlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:02.793898 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.793892    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8550012-8faa-4f06-9095-cd4aca5cb22e" (UID: "b8550012-8faa-4f06-9095-cd4aca5cb22e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:02.794057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.793912    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8550012-8faa-4f06-9095-cd4aca5cb22e-kube-api-access-p4vvp" (OuterVolumeSpecName: "kube-api-access-p4vvp") pod "b8550012-8faa-4f06-9095-cd4aca5cb22e" (UID: "b8550012-8faa-4f06-9095-cd4aca5cb22e"). InnerVolumeSpecName "kube-api-access-p4vvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:02.893236 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.893209    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8550012-8faa-4f06-9095-cd4aca5cb22e-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:02.893236 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.893232    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkqlh\" (UniqueName: \"kubernetes.io/projected/839ba90d-0bdc-4d94-9b1e-82987093c7cc-kube-api-access-lkqlh\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:02.893236 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.893242    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4vvp\" (UniqueName: \"kubernetes.io/projected/b8550012-8faa-4f06-9095-cd4aca5cb22e-kube-api-access-p4vvp\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:02.893458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.893251    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/839ba90d-0bdc-4d94-9b1e-82987093c7cc-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:02.893458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:02.893259    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8550012-8faa-4f06-9095-cd4aca5cb22e-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:03.456093 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.456062    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"
Jul 24 17:42:03.456536 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.456118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95" event={"ID":"839ba90d-0bdc-4d94-9b1e-82987093c7cc","Type":"ContainerDied","Data":"a360e75c05c0ecc1efe1f79aba92250229956caab6529f6a30ffddd07c39f364"}
Jul 24 17:42:03.456536 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.456177    2562 scope.go:117] "RemoveContainer" containerID="f7360b7571ec22713a10bfafbf67de9e1be30a754e8616daac65bc38e4b9df06"
Jul 24 17:42:03.457569 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.457391    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"
Jul 24 17:42:03.457569 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.457403    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd" event={"ID":"a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a","Type":"ContainerDied","Data":"27ace0906576649a007c69995e1c12f6bab82eae5fc353910537df9fb1120aac"}
Jul 24 17:42:03.458830 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.458808    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg" event={"ID":"b8550012-8faa-4f06-9095-cd4aca5cb22e","Type":"ContainerDied","Data":"ce1dec1885a1594fc773a95f9d2b6f3164ae96045e0c34c4099465965ada7b16"}
Jul 24 17:42:03.458946 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.458857    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"
Jul 24 17:42:03.470158 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.470137    2562 scope.go:117] "RemoveContainer" containerID="4ab8f695e50f1c552014d59acb3b9efd5491a5f9b88490c8b024eac095354f8f"
Jul 24 17:42:03.477750 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.477733    2562 scope.go:117] "RemoveContainer" containerID="8b7fc3745ae5d85af637bbd8c4a84c63eb94dc1661332af9f004ac1e68be567c"
Jul 24 17:42:03.486053 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.486032    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"]
Jul 24 17:42:03.490989 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.490968    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-cccec6a6-6c7cc6d6f6-mbk95"]
Jul 24 17:42:03.509564 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.504318    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="839ba90d-0bdc-4d94-9b1e-82987093c7cc" path="/var/lib/kubelet/pods/839ba90d-0bdc-4d94-9b1e-82987093c7cc/volumes"
Jul 24 17:42:03.509564 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.505112    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"]
Jul 24 17:42:03.509564 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.506591    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-cccec6a6-796f5c7494-g2kzd"]
Jul 24 17:42:03.519966 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.519903    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"]
Jul 24 17:42:03.523763 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:03.523742    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-cccec6a6-5bb45b6854-xn7jg"]
Jul 24 17:42:05.504590 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:05.504557    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a" path="/var/lib/kubelet/pods/a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a/volumes"
Jul 24 17:42:05.504965 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:05.504904    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8550012-8faa-4f06-9095-cd4aca5cb22e" path="/var/lib/kubelet/pods/b8550012-8faa-4f06-9095-cd4aca5cb22e/volumes"
Jul 24 17:42:13.654631 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:13.654597    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"]
Jul 24 17:42:13.655136 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:13.654854    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy" containerID="cri-o://68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992" gracePeriod=30
Jul 24 17:42:18.911184 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:18.911155    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:42:19.021361 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021333    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-envoy\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021361 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021364    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-credential-socket\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021389    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-data\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021404    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istiod-ca-cert\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021427    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xd76\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-kube-api-access-6xd76\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021455    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-token\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021516    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-certs\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021552    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-podinfo\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021573    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-socket\") pod \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\" (UID: \"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c\") "
Jul 24 17:42:19.021943 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021622    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:19.021943 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021826    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.021943 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021828    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:19.021943 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.021885    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:19.022187 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.022051    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:19.022187 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.022173    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:19.023850 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.023822    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:19.023976 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.023953    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:19.024088 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.024040    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-kube-api-access-6xd76" (OuterVolumeSpecName: "kube-api-access-6xd76") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "kube-api-access-6xd76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:19.024141 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.024105    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" (UID: "4ad21d6d-6127-4ea3-b8e2-0f7155354f8c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:42:19.122266 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.122237    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.122266 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.122261    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.122441 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.122273    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.122441 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.122285    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.122441 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.122295    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.122441 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.122305    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.122441 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.122316    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xd76\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-kube-api-access-6xd76\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.122441 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.122328    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:42:19.527574 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.527536    2562 generic.go:356] "Generic (PLEG): container finished" podID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerID="68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992" exitCode=0
Jul 24 17:42:19.527741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.527600    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"
Jul 24 17:42:19.527741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.527612    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" event={"ID":"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c","Type":"ContainerDied","Data":"68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992"}
Jul 24 17:42:19.527741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.527662    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm" event={"ID":"4ad21d6d-6127-4ea3-b8e2-0f7155354f8c","Type":"ContainerDied","Data":"dc21e910ae4a9da71532fae07329921d7bf2771f82c617d357dfc08cc34ebe6b"}
Jul 24 17:42:19.527741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.527683    2562 scope.go:117] "RemoveContainer" containerID="68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992"
Jul 24 17:42:19.535849 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.535832    2562 scope.go:117] "RemoveContainer" containerID="d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8"
Jul 24 17:42:19.543744 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.543724    2562 scope.go:117] "RemoveContainer" containerID="68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992"
Jul 24 17:42:19.544002 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:42:19.543984    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992\": container with ID starting with 68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992 not found: ID does not exist" containerID="68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992"
Jul 24 17:42:19.544180 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.544011    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992"} err="failed to get container status \"68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992\": rpc error: code = NotFound desc = could not find container \"68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992\": container with ID starting with 68805110c63731c24638cf19d1979eb646cb60b17fa7bf690b56737bf999b992 not found: ID does not exist"
Jul 24 17:42:19.544180 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.544056    2562 scope.go:117] "RemoveContainer" containerID="d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8"
Jul 24 17:42:19.544351 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:42:19.544334    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8\": container with ID starting with d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8 not found: ID does not exist" containerID="d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8"
Jul 24 17:42:19.544404 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.544360    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8"} err="failed to get container status \"d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8\": rpc error: code = NotFound desc = could not find container \"d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8\": container with ID starting with d94c6eba0d448c54ad56f0c647c707019521fa6735d710452af5004d20c5e4f8 not found: ID does not exist"
Jul 24 17:42:19.544581 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.544565    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"]
Jul 24 17:42:19.549069 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:19.549049    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-cccec6a6-openshift-default-5b796f77b8-bxsgm"]
Jul 24 17:42:21.504669 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:21.504635    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" path="/var/lib/kubelet/pods/4ad21d6d-6127-4ea3-b8e2-0f7155354f8c/volumes"
Jul 24 17:42:32.854831 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.854795    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"]
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855127    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8550012-8faa-4f06-9095-cd4aca5cb22e" containerName="maas-api"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855140    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8550012-8faa-4f06-9095-cd4aca5cb22e" containerName="maas-api"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855159    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855165    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855175    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="839ba90d-0bdc-4d94-9b1e-82987093c7cc" containerName="payload-pre-processing"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855180    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="839ba90d-0bdc-4d94-9b1e-82987093c7cc" containerName="payload-pre-processing"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855186    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a" containerName="payload-processing"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855191    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a" containerName="payload-processing"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855197    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy"
Jul 24 17:42:32.855237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855203    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy"
Jul 24 17:42:32.855570 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855248    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3c8eeb2-4f06-4e4b-a3e6-3dc6e39ab11a" containerName="payload-processing"
Jul 24 17:42:32.855570 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855255    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8550012-8faa-4f06-9095-cd4aca5cb22e" containerName="maas-api"
Jul 24 17:42:32.855570 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855261    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy"
Jul 24 17:42:32.855570 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855267    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ad21d6d-6127-4ea3-b8e2-0f7155354f8c" containerName="istio-proxy"
Jul 24 17:42:32.855570 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.855273    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="839ba90d-0bdc-4d94-9b1e-82987093c7cc" containerName="payload-pre-processing"
Jul 24 17:42:32.859661 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.859643    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.862402 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.862370    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a5726951-openshift-default-dockercfg-977x8\""
Jul 24 17:42:32.869845 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.869813    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"]
Jul 24 17:42:32.942701 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.942660    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-socket\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.942892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.942734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-token\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.942892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.942763    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqkl5\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-kube-api-access-pqkl5\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.942892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.942792    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-certs\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.942892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.942825    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-data\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.943094 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.942918    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-envoy\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.943094 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.942949    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-credential-socket\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.943094 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.943003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc69bb5-6472-49c6-af19-f63552e1cc42-istiod-ca-cert\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:32.943094 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:32.943055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-podinfo\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.043885 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.043850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-certs\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.043898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-data\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.043939    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-envoy\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.043957    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-credential-socket\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.043995    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc69bb5-6472-49c6-af19-f63552e1cc42-istiod-ca-cert\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-podinfo\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-socket\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044116    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-token\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044477 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044138    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqkl5\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-kube-api-access-pqkl5\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044477 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044384    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-certs\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044601 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044490    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-data\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044601 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044525    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-socket\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044784 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-credential-socket\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.044907 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.044887    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc69bb5-6472-49c6-af19-f63552e1cc42-istiod-ca-cert\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.047200 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.047174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-envoy\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.047382 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.047363    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-podinfo\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.052376 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.052357    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-token\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.052477 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.052462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqkl5\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-kube-api-access-pqkl5\") pod \"e2e-mt-a5726951-openshift-default-64656bbf56-nrg76\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") " pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.171156 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.171069    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:33.304717 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.304693    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"]
Jul 24 17:42:33.307334 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:42:33.307309    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc69bb5_6472_49c6_af19_f63552e1cc42.slice/crio-fe6afebecd896442afafbfa39031d9f1a85975da55dcef6b041678dd6173ef31 WatchSource:0}: Error finding container fe6afebecd896442afafbfa39031d9f1a85975da55dcef6b041678dd6173ef31: Status 404 returned error can't find the container with id fe6afebecd896442afafbfa39031d9f1a85975da55dcef6b041678dd6173ef31
Jul 24 17:42:33.309456 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.309418    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:42:33.309527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.309506    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:42:33.309564 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.309546    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:42:33.586453 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.586422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" event={"ID":"9bc69bb5-6472-49c6-af19-f63552e1cc42","Type":"ContainerStarted","Data":"fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7"}
Jul 24 17:42:33.586636 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.586470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" event={"ID":"9bc69bb5-6472-49c6-af19-f63552e1cc42","Type":"ContainerStarted","Data":"fe6afebecd896442afafbfa39031d9f1a85975da55dcef6b041678dd6173ef31"}
Jul 24 17:42:33.610904 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:33.610858    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" podStartSLOduration=1.6108440979999998 podStartE2EDuration="1.610844098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:42:33.60861526 +0000 UTC m=+1942.579747443" watchObservedRunningTime="2026-07-24 17:42:33.610844098 +0000 UTC m=+1942.581976278"
Jul 24 17:42:34.172075 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:34.172038    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:34.177534 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:34.177510    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:34.591268 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:34.591240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:34.592199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:34.592182    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:35.727039 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.726987    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"]
Jul 24 17:42:35.730527 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.730504    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:35.733303 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.733276    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:42:35.733445 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.733304    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:42:35.733445 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.733336    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a5726951-serving-cert\""
Jul 24 17:42:35.734550 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.734531    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:42:35.741109 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.741082    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"]
Jul 24 17:42:35.867504 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.867462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:35.867702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.867515    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj6tc\" (UniqueName: \"kubernetes.io/projected/2f4d3b98-4777-499e-8978-aa1df49de0d3-kube-api-access-zj6tc\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:35.867702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.867614    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4d3b98-4777-499e-8978-aa1df49de0d3-openshift-service-ca\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:35.968250 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.968212    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4d3b98-4777-499e-8978-aa1df49de0d3-openshift-service-ca\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:35.968438 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.968285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:35.968438 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.968308    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj6tc\" (UniqueName: \"kubernetes.io/projected/2f4d3b98-4777-499e-8978-aa1df49de0d3-kube-api-access-zj6tc\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:35.968517 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:42:35.968460    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-serving-cert: secret "maas-api-e2e-mt-a5726951-serving-cert" not found
Jul 24 17:42:35.968555 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:42:35.968542    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls podName:2f4d3b98-4777-499e-8978-aa1df49de0d3 nodeName:}" failed. No retries permitted until 2026-07-24 17:42:36.468519958 +0000 UTC m=+1945.439652123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls") pod "maas-api-e2e-mt-a5726951-69ff6b9954-msm74" (UID: "2f4d3b98-4777-499e-8978-aa1df49de0d3") : secret "maas-api-e2e-mt-a5726951-serving-cert" not found
Jul 24 17:42:35.968870 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.968852    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4d3b98-4777-499e-8978-aa1df49de0d3-openshift-service-ca\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:35.980084 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:35.980001    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj6tc\" (UniqueName: \"kubernetes.io/projected/2f4d3b98-4777-499e-8978-aa1df49de0d3-kube-api-access-zj6tc\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:36.332274 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.332187    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"]
Jul 24 17:42:36.335611 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.335594    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:36.338222 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.338202    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-a5726951-dockercfg-5gtld\""
Jul 24 17:42:36.338348 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.338247    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a5726951\""
Jul 24 17:42:36.347054 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.347030    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"]
Jul 24 17:42:36.471528 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.471490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:36.471752 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.471589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbsgj\" (UniqueName: \"kubernetes.io/projected/c7d684b3-80e3-4bb4-9999-b9897f426f4c-kube-api-access-tbsgj\") pod \"payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm\" (UID: \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\") " pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:36.471752 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.471629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7d684b3-80e3-4bb4-9999-b9897f426f4c-plugins-config-volume\") pod \"payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm\" (UID: \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\") " pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:36.474365 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.474336    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls\") pod \"maas-api-e2e-mt-a5726951-69ff6b9954-msm74\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:36.572480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.572436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbsgj\" (UniqueName: \"kubernetes.io/projected/c7d684b3-80e3-4bb4-9999-b9897f426f4c-kube-api-access-tbsgj\") pod \"payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm\" (UID: \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\") " pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:36.572480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.572482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7d684b3-80e3-4bb4-9999-b9897f426f4c-plugins-config-volume\") pod \"payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm\" (UID: \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\") " pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:36.573291 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.573272    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7d684b3-80e3-4bb4-9999-b9897f426f4c-plugins-config-volume\") pod \"payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm\" (UID: \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\") " pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:36.581000 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.580975    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbsgj\" (UniqueName: \"kubernetes.io/projected/c7d684b3-80e3-4bb4-9999-b9897f426f4c-kube-api-access-tbsgj\") pod \"payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm\" (UID: \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\") " pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:36.642116 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.642004    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:36.646217 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.646193    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:36.793796 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.793535    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"]
Jul 24 17:42:36.795796 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:42:36.795759    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4d3b98_4777_499e_8978_aa1df49de0d3.slice/crio-7e3aedd82384979b35754481fd06c947a5aa6fabe2394be6a81a2eb7c152f06e WatchSource:0}: Error finding container 7e3aedd82384979b35754481fd06c947a5aa6fabe2394be6a81a2eb7c152f06e: Status 404 returned error can't find the container with id 7e3aedd82384979b35754481fd06c947a5aa6fabe2394be6a81a2eb7c152f06e
Jul 24 17:42:36.797399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.797384    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:42:36.811969 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:36.811944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"]
Jul 24 17:42:36.814223 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:42:36.814200    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d684b3_80e3_4bb4_9999_b9897f426f4c.slice/crio-ca6d3032612146c9ac0688dd444efac4b82ef23808f9d2cfcfc7e500f9b27688 WatchSource:0}: Error finding container ca6d3032612146c9ac0688dd444efac4b82ef23808f9d2cfcfc7e500f9b27688: Status 404 returned error can't find the container with id ca6d3032612146c9ac0688dd444efac4b82ef23808f9d2cfcfc7e500f9b27688
Jul 24 17:42:37.604912 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:37.604873    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm" event={"ID":"c7d684b3-80e3-4bb4-9999-b9897f426f4c","Type":"ContainerStarted","Data":"88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d"}
Jul 24 17:42:37.605157 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:37.604929    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm" event={"ID":"c7d684b3-80e3-4bb4-9999-b9897f426f4c","Type":"ContainerStarted","Data":"ca6d3032612146c9ac0688dd444efac4b82ef23808f9d2cfcfc7e500f9b27688"}
Jul 24 17:42:37.605157 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:37.604997    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:37.606432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:37.606407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74" event={"ID":"2f4d3b98-4777-499e-8978-aa1df49de0d3","Type":"ContainerStarted","Data":"5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd"}
Jul 24 17:42:37.606536 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:37.606446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74" event={"ID":"2f4d3b98-4777-499e-8978-aa1df49de0d3","Type":"ContainerStarted","Data":"7e3aedd82384979b35754481fd06c947a5aa6fabe2394be6a81a2eb7c152f06e"}
Jul 24 17:42:37.606536 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:37.606472    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:37.622604 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:37.622554    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm" podStartSLOduration=1.622539747 podStartE2EDuration="1.622539747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:42:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:42:37.620806029 +0000 UTC m=+1946.591938211" watchObservedRunningTime="2026-07-24 17:42:37.622539747 +0000 UTC m=+1946.593671936"
Jul 24 17:42:37.641656 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:37.641607    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74" podStartSLOduration=2.294471113 podStartE2EDuration="2.641590921s" totalImagesPullingTime="347.119808ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:42:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:42:37.640679695 +0000 UTC m=+1946.611811876" watchObservedRunningTime="2026-07-24 17:42:37.641590921 +0000 UTC m=+1946.612723131"
Jul 24 17:42:43.616323 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:43.616296    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:42:46.644983 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:46.644947    2562 generic.go:356] "Generic (PLEG): container finished" podID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerID="fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7" exitCode=0
Jul 24 17:42:46.645512 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:46.645044    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" event={"ID":"9bc69bb5-6472-49c6-af19-f63552e1cc42","Type":"ContainerDied","Data":"fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7"}
Jul 24 17:42:46.645512 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:46.645439    2562 scope.go:117] "RemoveContainer" containerID="fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7"
Jul 24 17:42:46.646485 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:46.646455    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:42:46.646580 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:46.646539    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:42:46.646642 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:46.646585    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:42:47.652279 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:47.652244    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" event={"ID":"9bc69bb5-6472-49c6-af19-f63552e1cc42","Type":"ContainerStarted","Data":"541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472"}
Jul 24 17:42:48.172896 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:48.172851    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:48.173149 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:48.173067    2562 patch_prober.go:28] interesting pod/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 24 17:42:48.173149 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:48.173116    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 24 17:42:48.612509 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:48.612478    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:42:49.172378 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:49.172341    2562 patch_prober.go:28] interesting pod/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 24 17:42:49.173045 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:49.172994    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 24 17:42:49.592041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:49.591997    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:50.172548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:50.172507    2562 patch_prober.go:28] interesting pod/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 24 17:42:50.173191 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:50.172586    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 24 17:42:51.172177 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:51.172130    2562 patch_prober.go:28] interesting pod/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 24 17:42:51.172523 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:51.172466    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 24 17:42:52.175835 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:52.175802    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:42:52.176728 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:42:52.176710    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:43:39.955679 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:39.955645    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"]
Jul 24 17:43:39.956276 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:39.955864    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74" podUID="2f4d3b98-4777-499e-8978-aa1df49de0d3" containerName="maas-api" containerID="cri-o://5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd" gracePeriod=30
Jul 24 17:43:40.011133 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.011101    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"]
Jul 24 17:43:40.011423 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.011358    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm" podUID="c7d684b3-80e3-4bb4-9999-b9897f426f4c" containerName="payload-processing" containerID="cri-o://88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d" gracePeriod=30
Jul 24 17:43:40.283070 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.283041    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:43:40.316693 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.316667    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:43:40.424654 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.424619    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls\") pod \"2f4d3b98-4777-499e-8978-aa1df49de0d3\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") "
Jul 24 17:43:40.424851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.424695    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7d684b3-80e3-4bb4-9999-b9897f426f4c-plugins-config-volume\") pod \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\" (UID: \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\") "
Jul 24 17:43:40.424851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.424741    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbsgj\" (UniqueName: \"kubernetes.io/projected/c7d684b3-80e3-4bb4-9999-b9897f426f4c-kube-api-access-tbsgj\") pod \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\" (UID: \"c7d684b3-80e3-4bb4-9999-b9897f426f4c\") "
Jul 24 17:43:40.424851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.424774    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4d3b98-4777-499e-8978-aa1df49de0d3-openshift-service-ca\") pod \"2f4d3b98-4777-499e-8978-aa1df49de0d3\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") "
Jul 24 17:43:40.424851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.424825    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zj6tc\" (UniqueName: \"kubernetes.io/projected/2f4d3b98-4777-499e-8978-aa1df49de0d3-kube-api-access-zj6tc\") pod \"2f4d3b98-4777-499e-8978-aa1df49de0d3\" (UID: \"2f4d3b98-4777-499e-8978-aa1df49de0d3\") "
Jul 24 17:43:40.425135 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.425106    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7d684b3-80e3-4bb4-9999-b9897f426f4c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7d684b3-80e3-4bb4-9999-b9897f426f4c" (UID: "c7d684b3-80e3-4bb4-9999-b9897f426f4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:43:40.425197 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.425181    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4d3b98-4777-499e-8978-aa1df49de0d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f4d3b98-4777-499e-8978-aa1df49de0d3" (UID: "2f4d3b98-4777-499e-8978-aa1df49de0d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:43:40.427032 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.426993    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7d684b3-80e3-4bb4-9999-b9897f426f4c-kube-api-access-tbsgj" (OuterVolumeSpecName: "kube-api-access-tbsgj") pod "c7d684b3-80e3-4bb4-9999-b9897f426f4c" (UID: "c7d684b3-80e3-4bb4-9999-b9897f426f4c"). InnerVolumeSpecName "kube-api-access-tbsgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:43:40.427032 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.426996    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f4d3b98-4777-499e-8978-aa1df49de0d3" (UID: "2f4d3b98-4777-499e-8978-aa1df49de0d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:43:40.427153 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.427005    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f4d3b98-4777-499e-8978-aa1df49de0d3-kube-api-access-zj6tc" (OuterVolumeSpecName: "kube-api-access-zj6tc") pod "2f4d3b98-4777-499e-8978-aa1df49de0d3" (UID: "2f4d3b98-4777-499e-8978-aa1df49de0d3"). InnerVolumeSpecName "kube-api-access-zj6tc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:43:40.526476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.526391    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f4d3b98-4777-499e-8978-aa1df49de0d3-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:43:40.526476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.526419    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7d684b3-80e3-4bb4-9999-b9897f426f4c-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:43:40.526476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.526430    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbsgj\" (UniqueName: \"kubernetes.io/projected/c7d684b3-80e3-4bb4-9999-b9897f426f4c-kube-api-access-tbsgj\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:43:40.526476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.526441    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f4d3b98-4777-499e-8978-aa1df49de0d3-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:43:40.526476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.526449    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj6tc\" (UniqueName: \"kubernetes.io/projected/2f4d3b98-4777-499e-8978-aa1df49de0d3-kube-api-access-zj6tc\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:43:40.887089 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.886973    2562 generic.go:356] "Generic (PLEG): container finished" podID="c7d684b3-80e3-4bb4-9999-b9897f426f4c" containerID="88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d" exitCode=0
Jul 24 17:43:40.887089 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.887031    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm" event={"ID":"c7d684b3-80e3-4bb4-9999-b9897f426f4c","Type":"ContainerDied","Data":"88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d"}
Jul 24 17:43:40.887089 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.887046    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"
Jul 24 17:43:40.887356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.887094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm" event={"ID":"c7d684b3-80e3-4bb4-9999-b9897f426f4c","Type":"ContainerDied","Data":"ca6d3032612146c9ac0688dd444efac4b82ef23808f9d2cfcfc7e500f9b27688"}
Jul 24 17:43:40.887356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.887119    2562 scope.go:117] "RemoveContainer" containerID="88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d"
Jul 24 17:43:40.888405 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.888383    2562 generic.go:356] "Generic (PLEG): container finished" podID="2f4d3b98-4777-499e-8978-aa1df49de0d3" containerID="5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd" exitCode=0
Jul 24 17:43:40.888513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.888428    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"
Jul 24 17:43:40.888513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.888443    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74" event={"ID":"2f4d3b98-4777-499e-8978-aa1df49de0d3","Type":"ContainerDied","Data":"5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd"}
Jul 24 17:43:40.888513 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.888477    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74" event={"ID":"2f4d3b98-4777-499e-8978-aa1df49de0d3","Type":"ContainerDied","Data":"7e3aedd82384979b35754481fd06c947a5aa6fabe2394be6a81a2eb7c152f06e"}
Jul 24 17:43:40.896006 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.895988    2562 scope.go:117] "RemoveContainer" containerID="88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d"
Jul 24 17:43:40.896324 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:43:40.896302    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d\": container with ID starting with 88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d not found: ID does not exist" containerID="88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d"
Jul 24 17:43:40.896407 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.896333    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d"} err="failed to get container status \"88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d\": rpc error: code = NotFound desc = could not find container \"88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d\": container with ID starting with 88c548c3a6cc1b0e7b8d2dd17e42ef45dac0f09e435ac620251cb8d09e5d774d not found: ID does not exist"
Jul 24 17:43:40.896407 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.896349    2562 scope.go:117] "RemoveContainer" containerID="5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd"
Jul 24 17:43:40.904299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.904280    2562 scope.go:117] "RemoveContainer" containerID="5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd"
Jul 24 17:43:40.904551 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:43:40.904534    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd\": container with ID starting with 5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd not found: ID does not exist" containerID="5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd"
Jul 24 17:43:40.904624 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.904564    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd"} err="failed to get container status \"5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd\": rpc error: code = NotFound desc = could not find container \"5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd\": container with ID starting with 5346fb6b6c0354bddf8cffb8eea77e7153b3fcea880a4fde7395fd33ed57d3bd not found: ID does not exist"
Jul 24 17:43:40.912113 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.912091    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"]
Jul 24 17:43:40.918500 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.918476    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a5726951-69ff6b9954-msm74"]
Jul 24 17:43:40.934760 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.934730    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"]
Jul 24 17:43:40.936064 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:40.936044    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a5726951-59cf9b5dc4-xr5fm"]
Jul 24 17:43:41.505232 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:41.505201    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f4d3b98-4777-499e-8978-aa1df49de0d3" path="/var/lib/kubelet/pods/2f4d3b98-4777-499e-8978-aa1df49de0d3/volumes"
Jul 24 17:43:41.505609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:43:41.505524    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7d684b3-80e3-4bb4-9999-b9897f426f4c" path="/var/lib/kubelet/pods/c7d684b3-80e3-4bb4-9999-b9897f426f4c/volumes"
Jul 24 17:44:35.596678 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:35.596591    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"]
Jul 24 17:44:35.597218 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:35.596801    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy" containerID="cri-o://541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472" gracePeriod=30
Jul 24 17:44:37.178004 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:37.177967    2562 patch_prober.go:28] interesting pod/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 17:44:37.178394 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:37.178046    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 17:44:40.850005 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.849974    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:44:40.913247 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913162    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-token\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913247 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913210    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqkl5\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-kube-api-access-pqkl5\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913247 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913236    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc69bb5-6472-49c6-af19-f63552e1cc42-istiod-ca-cert\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913282    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-socket\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913323    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-credential-socket\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913372    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-certs\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913409    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-envoy\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913432    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-podinfo\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913522 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913456    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-data\") pod \"9bc69bb5-6472-49c6-af19-f63552e1cc42\" (UID: \"9bc69bb5-6472-49c6-af19-f63552e1cc42\") "
Jul 24 17:44:40.913836 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913567    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:40.913836 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913669    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bc69bb5-6472-49c6-af19-f63552e1cc42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:44:40.913836 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913830    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bc69bb5-6472-49c6-af19-f63552e1cc42-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:40.914001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913849    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:40.914001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.913881    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:40.914273 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.914245    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:40.914571 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.914546    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-data" (OuterVolumeSpecName: "istio-data") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:40.915607 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.915585    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-token" (OuterVolumeSpecName: "istio-token") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:44:40.915734 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.915701    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-kube-api-access-pqkl5" (OuterVolumeSpecName: "kube-api-access-pqkl5") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "kube-api-access-pqkl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:44:40.915905 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.915873    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:44:40.915998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:40.915943    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9bc69bb5-6472-49c6-af19-f63552e1cc42" (UID: "9bc69bb5-6472-49c6-af19-f63552e1cc42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:44:41.014545 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.014512    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:41.014545 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.014539    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:41.014545 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.014549    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:41.014793 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.014556    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:41.014793 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.014565    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqkl5\" (UniqueName: \"kubernetes.io/projected/9bc69bb5-6472-49c6-af19-f63552e1cc42-kube-api-access-pqkl5\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:41.014793 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.014575    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:41.014793 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.014583    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bc69bb5-6472-49c6-af19-f63552e1cc42-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:44:41.145665 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.145631    2562 generic.go:356] "Generic (PLEG): container finished" podID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerID="541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472" exitCode=0
Jul 24 17:44:41.145851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.145682    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"
Jul 24 17:44:41.145851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.145689    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" event={"ID":"9bc69bb5-6472-49c6-af19-f63552e1cc42","Type":"ContainerDied","Data":"541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472"}
Jul 24 17:44:41.145851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.145763    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76" event={"ID":"9bc69bb5-6472-49c6-af19-f63552e1cc42","Type":"ContainerDied","Data":"fe6afebecd896442afafbfa39031d9f1a85975da55dcef6b041678dd6173ef31"}
Jul 24 17:44:41.145851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.145785    2562 scope.go:117] "RemoveContainer" containerID="541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472"
Jul 24 17:44:41.154869 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.154847    2562 scope.go:117] "RemoveContainer" containerID="fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7"
Jul 24 17:44:41.162478 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.162462    2562 scope.go:117] "RemoveContainer" containerID="541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472"
Jul 24 17:44:41.162730 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:44:41.162711    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472\": container with ID starting with 541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472 not found: ID does not exist" containerID="541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472"
Jul 24 17:44:41.162807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.162738    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472"} err="failed to get container status \"541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472\": rpc error: code = NotFound desc = could not find container \"541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472\": container with ID starting with 541ec82ae9996e2be51ab83d1d360b58dd91e2be8945368bf4e682bdd52b4472 not found: ID does not exist"
Jul 24 17:44:41.162807 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.162753    2562 scope.go:117] "RemoveContainer" containerID="fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7"
Jul 24 17:44:41.162995 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:44:41.162978    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7\": container with ID starting with fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7 not found: ID does not exist" containerID="fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7"
Jul 24 17:44:41.163068 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.163004    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7"} err="failed to get container status \"fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7\": rpc error: code = NotFound desc = could not find container \"fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7\": container with ID starting with fbcf1c2e378f2510a8eb5e1591bbf10dccaff737106684fb8d00ad3b5aa522c7 not found: ID does not exist"
Jul 24 17:44:41.170424 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.170365    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"]
Jul 24 17:44:41.174291 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.174270    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a5726951-openshift-default-64656bbf56-nrg76"]
Jul 24 17:44:41.506225 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:44:41.506193    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" path="/var/lib/kubelet/pods/9bc69bb5-6472-49c6-af19-f63552e1cc42/volumes"
Jul 24 17:45:00.132214 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132182    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn"]
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132475    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7d684b3-80e3-4bb4-9999-b9897f426f4c" containerName="payload-processing"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132486    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d684b3-80e3-4bb4-9999-b9897f426f4c" containerName="payload-processing"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132497    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132502    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132516    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132522    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132529    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4d3b98-4777-499e-8978-aa1df49de0d3" containerName="maas-api"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132534    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4d3b98-4777-499e-8978-aa1df49de0d3" containerName="maas-api"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132599    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4d3b98-4777-499e-8978-aa1df49de0d3" containerName="maas-api"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132607    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132613    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bc69bb5-6472-49c6-af19-f63552e1cc42" containerName="istio-proxy"
Jul 24 17:45:00.132736 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.132620    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7d684b3-80e3-4bb4-9999-b9897f426f4c" containerName="payload-processing"
Jul 24 17:45:00.134946 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.134928    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn"
Jul 24 17:45:00.137685 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.137667    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-d6flx\""
Jul 24 17:45:00.143180 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.143156    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn"]
Jul 24 17:45:00.146544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.146516    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn"
Jul 24 17:45:00.289369 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:00.289324    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn"]
Jul 24 17:45:00.291519 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:45:00.291483    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42cd707_38b8_4e45_9ad8_29fb1b351331.slice/crio-4b219a4b2bbbc151ff573cfa281e7f5d0b6fd1ed31e347559b972696a632d4c0 WatchSource:0}: Error finding container 4b219a4b2bbbc151ff573cfa281e7f5d0b6fd1ed31e347559b972696a632d4c0: Status 404 returned error can't find the container with id 4b219a4b2bbbc151ff573cfa281e7f5d0b6fd1ed31e347559b972696a632d4c0
Jul 24 17:45:01.232322 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:01.232283    2562 generic.go:356] "Generic (PLEG): container finished" podID="b42cd707-38b8-4e45-9ad8-29fb1b351331" containerID="69b8512a99c77714be58bd97dae4698a4bba27772d2aaea62e69c1ae1c5d79b8" exitCode=0
Jul 24 17:45:01.232726 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:01.232367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn" event={"ID":"b42cd707-38b8-4e45-9ad8-29fb1b351331","Type":"ContainerDied","Data":"69b8512a99c77714be58bd97dae4698a4bba27772d2aaea62e69c1ae1c5d79b8"}
Jul 24 17:45:01.232726 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:01.232423    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn" event={"ID":"b42cd707-38b8-4e45-9ad8-29fb1b351331","Type":"ContainerStarted","Data":"4b219a4b2bbbc151ff573cfa281e7f5d0b6fd1ed31e347559b972696a632d4c0"}
Jul 24 17:45:02.366664 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:02.366637    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn"
Jul 24 17:45:03.243112 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:03.243076    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn"
Jul 24 17:45:03.243112 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:03.243093    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748585-ptbnn" event={"ID":"b42cd707-38b8-4e45-9ad8-29fb1b351331","Type":"ContainerDied","Data":"4b219a4b2bbbc151ff573cfa281e7f5d0b6fd1ed31e347559b972696a632d4c0"}
Jul 24 17:45:03.243328 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:03.243138    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b219a4b2bbbc151ff573cfa281e7f5d0b6fd1ed31e347559b972696a632d4c0"
Jul 24 17:45:11.554201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:11.554175    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:45:11.557403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:11.557384    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:45:30.897081 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.897045    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"]
Jul 24 17:45:30.897518 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.897462    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b42cd707-38b8-4e45-9ad8-29fb1b351331" containerName="cleanup"
Jul 24 17:45:30.897518 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.897476    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b42cd707-38b8-4e45-9ad8-29fb1b351331" containerName="cleanup"
Jul 24 17:45:30.897599 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.897566    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b42cd707-38b8-4e45-9ad8-29fb1b351331" containerName="cleanup"
Jul 24 17:45:30.900645 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.900625    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.903326 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.903304    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-387e6e-openshift-default-dockercfg-sktl4\""
Jul 24 17:45:30.911619 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.911598    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"]
Jul 24 17:45:30.925491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925468    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8wvn\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-kube-api-access-g8wvn\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.925630 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925508    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581c826c-f536-4569-b2c3-c6156fbb000d-istio-podinfo\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.925630 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925531    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-credential-socket\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.925630 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925590    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581c826c-f536-4569-b2c3-c6156fbb000d-istiod-ca-cert\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.925804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925642    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-socket\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.925804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925674    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-data\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.925804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-istio-token\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.925804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925772    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-envoy\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:30.925804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:30.925798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-certs\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.026853 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.026813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581c826c-f536-4569-b2c3-c6156fbb000d-istio-podinfo\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.026860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-credential-socket\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.026975    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581c826c-f536-4569-b2c3-c6156fbb000d-istiod-ca-cert\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-socket\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027079 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-data\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027329 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027141    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-istio-token\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027329 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-envoy\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027329 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027203    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-certs\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027329 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8wvn\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-kube-api-access-g8wvn\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027329 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027305    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-credential-socket\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027598 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-socket\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.027753 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027728    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581c826c-f536-4569-b2c3-c6156fbb000d-istiod-ca-cert\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.028004 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-data\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.028004 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.027733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-certs\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.029550 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.029532    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-envoy\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.029953 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.029932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581c826c-f536-4569-b2c3-c6156fbb000d-istio-podinfo\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.035487 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.035468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-istio-token\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.035756 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.035736    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8wvn\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-kube-api-access-g8wvn\") pod \"e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") " pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.212238 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.212208    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:31.347935 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.347906    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"]
Jul 24 17:45:31.349743 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:45:31.349713    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581c826c_f536_4569_b2c3_c6156fbb000d.slice/crio-238bfafc863d712c805dc4bbb0bb451fe4d9ac091c419421655ad300dae59924 WatchSource:0}: Error finding container 238bfafc863d712c805dc4bbb0bb451fe4d9ac091c419421655ad300dae59924: Status 404 returned error can't find the container with id 238bfafc863d712c805dc4bbb0bb451fe4d9ac091c419421655ad300dae59924
Jul 24 17:45:31.352370 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.352329    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:31.352441 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.352410    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:31.352480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.352454    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:31.360698 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:31.360673    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" event={"ID":"581c826c-f536-4569-b2c3-c6156fbb000d","Type":"ContainerStarted","Data":"238bfafc863d712c805dc4bbb0bb451fe4d9ac091c419421655ad300dae59924"}
Jul 24 17:45:32.366563 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:32.366528    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" event={"ID":"581c826c-f536-4569-b2c3-c6156fbb000d","Type":"ContainerStarted","Data":"e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56"}
Jul 24 17:45:32.387808 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:32.387751    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" podStartSLOduration=2.387734083 podStartE2EDuration="2.387734083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:32.386331402 +0000 UTC m=+2121.357463583" watchObservedRunningTime="2026-07-24 17:45:32.387734083 +0000 UTC m=+2121.358866321"
Jul 24 17:45:33.213310 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:33.213271    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:33.218119 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:33.218092    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:33.370347 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:33.370312    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:33.371368 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:33.371350    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:34.500603 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.500555    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"]
Jul 24 17:45:34.504218 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.504196    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:34.507170 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.507139    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:45:34.507304 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.507189    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:45:34.508521 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.508503    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-387e6e-serving-cert\""
Jul 24 17:45:34.508658 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.508506    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:45:34.515691 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.515632    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"]
Jul 24 17:45:34.557751 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.557718    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:34.557935 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.557765    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecc0e12-eea3-495e-b021-594a3d760693-openshift-service-ca\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:34.557935 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.557789    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7csgn\" (UniqueName: \"kubernetes.io/projected/cecc0e12-eea3-495e-b021-594a3d760693-kube-api-access-7csgn\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:34.659113 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.659077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:34.659323 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.659120    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecc0e12-eea3-495e-b021-594a3d760693-openshift-service-ca\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:34.659323 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.659146    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7csgn\" (UniqueName: \"kubernetes.io/projected/cecc0e12-eea3-495e-b021-594a3d760693-kube-api-access-7csgn\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:34.659323 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:45:34.659243    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-serving-cert: secret "maas-api-e2e-inf-a-387e6e-serving-cert" not found
Jul 24 17:45:34.659323 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:45:34.659324    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls podName:cecc0e12-eea3-495e-b021-594a3d760693 nodeName:}" failed. No retries permitted until 2026-07-24 17:45:35.159303992 +0000 UTC m=+2124.130436168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls") pod "maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x" (UID: "cecc0e12-eea3-495e-b021-594a3d760693") : secret "maas-api-e2e-inf-a-387e6e-serving-cert" not found
Jul 24 17:45:34.659917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.659895    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecc0e12-eea3-495e-b021-594a3d760693-openshift-service-ca\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:34.668782 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:34.668752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7csgn\" (UniqueName: \"kubernetes.io/projected/cecc0e12-eea3-495e-b021-594a3d760693-kube-api-access-7csgn\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:35.138847 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.138810    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"]
Jul 24 17:45:35.142201 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.142160    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:35.144847 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.144825    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-387e6e\""
Jul 24 17:45:35.144964 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.144827    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-387e6e-dockercfg-lhqh6\""
Jul 24 17:45:35.153759 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.153737    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"]
Jul 24 17:45:35.164095 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.164071    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:35.166577 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.166558    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls\") pod \"maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:35.204047 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.203986    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"]
Jul 24 17:45:35.207299 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.207280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:35.219192 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.219169    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"]
Jul 24 17:45:35.264769 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.264739    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgkxw\" (UniqueName: \"kubernetes.io/projected/adb647dd-521c-469b-acb4-99154b8590a6-kube-api-access-pgkxw\") pod \"payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh\" (UID: \"adb647dd-521c-469b-acb4-99154b8590a6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:35.264769 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.264772    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4dcba02-0d8c-4456-b3f1-2704d70b415d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs\" (UID: \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:35.264975 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.264809    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtbt8\" (UniqueName: \"kubernetes.io/projected/e4dcba02-0d8c-4456-b3f1-2704d70b415d-kube-api-access-dtbt8\") pod \"payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs\" (UID: \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:35.264975 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.264872    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adb647dd-521c-469b-acb4-99154b8590a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh\" (UID: \"adb647dd-521c-469b-acb4-99154b8590a6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:35.366325 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.366288    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adb647dd-521c-469b-acb4-99154b8590a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh\" (UID: \"adb647dd-521c-469b-acb4-99154b8590a6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:35.366519 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.366380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgkxw\" (UniqueName: \"kubernetes.io/projected/adb647dd-521c-469b-acb4-99154b8590a6-kube-api-access-pgkxw\") pod \"payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh\" (UID: \"adb647dd-521c-469b-acb4-99154b8590a6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:35.366519 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.366399    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4dcba02-0d8c-4456-b3f1-2704d70b415d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs\" (UID: \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:35.366519 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.366457    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtbt8\" (UniqueName: \"kubernetes.io/projected/e4dcba02-0d8c-4456-b3f1-2704d70b415d-kube-api-access-dtbt8\") pod \"payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs\" (UID: \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:35.367077 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.367058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adb647dd-521c-469b-acb4-99154b8590a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh\" (UID: \"adb647dd-521c-469b-acb4-99154b8590a6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:35.367232 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.367217    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4dcba02-0d8c-4456-b3f1-2704d70b415d-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs\" (UID: \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:35.381521 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.381496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtbt8\" (UniqueName: \"kubernetes.io/projected/e4dcba02-0d8c-4456-b3f1-2704d70b415d-kube-api-access-dtbt8\") pod \"payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs\" (UID: \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\") " pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:35.381670 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.381652    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgkxw\" (UniqueName: \"kubernetes.io/projected/adb647dd-521c-469b-acb4-99154b8590a6-kube-api-access-pgkxw\") pod \"payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh\" (UID: \"adb647dd-521c-469b-acb4-99154b8590a6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:35.416528 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.416447    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:35.453945 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.453917    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:35.517693 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.517661    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:35.568228 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.567912    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"]
Jul 24 17:45:35.570988 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:45:35.570946    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcecc0e12_eea3_495e_b021_594a3d760693.slice/crio-b33c46685258763219bee589a91b566a72d3811d69fa0ce7040cb777143d0e8a WatchSource:0}: Error finding container b33c46685258763219bee589a91b566a72d3811d69fa0ce7040cb777143d0e8a: Status 404 returned error can't find the container with id b33c46685258763219bee589a91b566a72d3811d69fa0ce7040cb777143d0e8a
Jul 24 17:45:35.625811 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.625776    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"]
Jul 24 17:45:35.628918 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:45:35.628888    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4dcba02_0d8c_4456_b3f1_2704d70b415d.slice/crio-44202bb0e225706a21265f39291c0d01b4e8082799f36db2f9960fab0c80fa40 WatchSource:0}: Error finding container 44202bb0e225706a21265f39291c0d01b4e8082799f36db2f9960fab0c80fa40: Status 404 returned error can't find the container with id 44202bb0e225706a21265f39291c0d01b4e8082799f36db2f9960fab0c80fa40
Jul 24 17:45:35.675770 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:35.675747    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"]
Jul 24 17:45:35.677929 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:45:35.677901    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadb647dd_521c_469b_acb4_99154b8590a6.slice/crio-d230224781f5c8afa607edb713fda6b38c8948ddca5f178602f089395079a8eb WatchSource:0}: Error finding container d230224781f5c8afa607edb713fda6b38c8948ddca5f178602f089395079a8eb: Status 404 returned error can't find the container with id d230224781f5c8afa607edb713fda6b38c8948ddca5f178602f089395079a8eb
Jul 24 17:45:36.386098 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.386057    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs" event={"ID":"e4dcba02-0d8c-4456-b3f1-2704d70b415d","Type":"ContainerStarted","Data":"b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230"}
Jul 24 17:45:36.386336 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.386160    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs" event={"ID":"e4dcba02-0d8c-4456-b3f1-2704d70b415d","Type":"ContainerStarted","Data":"44202bb0e225706a21265f39291c0d01b4e8082799f36db2f9960fab0c80fa40"}
Jul 24 17:45:36.386336 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.386191    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:36.387897 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.387870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh" event={"ID":"adb647dd-521c-469b-acb4-99154b8590a6","Type":"ContainerStarted","Data":"c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e"}
Jul 24 17:45:36.388084 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.387925    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:36.388084 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.387951    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh" event={"ID":"adb647dd-521c-469b-acb4-99154b8590a6","Type":"ContainerStarted","Data":"d230224781f5c8afa607edb713fda6b38c8948ddca5f178602f089395079a8eb"}
Jul 24 17:45:36.389518 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.389481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x" event={"ID":"cecc0e12-eea3-495e-b021-594a3d760693","Type":"ContainerStarted","Data":"7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296"}
Jul 24 17:45:36.389629 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.389528    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x" event={"ID":"cecc0e12-eea3-495e-b021-594a3d760693","Type":"ContainerStarted","Data":"b33c46685258763219bee589a91b566a72d3811d69fa0ce7040cb777143d0e8a"}
Jul 24 17:45:36.389629 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.389570    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:36.403472 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.403426    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs" podStartSLOduration=1.403410804 podStartE2EDuration="1.403410804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:36.402276149 +0000 UTC m=+2125.373408331" watchObservedRunningTime="2026-07-24 17:45:36.403410804 +0000 UTC m=+2125.374542985"
Jul 24 17:45:36.419989 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.419933    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x" podStartSLOduration=1.7828637619999999 podStartE2EDuration="2.419916919s" totalImagesPullingTime="637.053157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:36.418376702 +0000 UTC m=+2125.389508897" watchObservedRunningTime="2026-07-24 17:45:36.419916919 +0000 UTC m=+2125.391049100"
Jul 24 17:45:36.447138 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:36.447066    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh" podStartSLOduration=1.447048038 podStartE2EDuration="1.447048038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:36.445901444 +0000 UTC m=+2125.417033625" watchObservedRunningTime="2026-07-24 17:45:36.447048038 +0000 UTC m=+2125.418180224"
Jul 24 17:45:39.388009 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.387976    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"]
Jul 24 17:45:39.391656 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.391635    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.394216 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.394197    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-22bf5f-openshift-default-dockercfg-pb5qz\""
Jul 24 17:45:39.403856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.403831    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"]
Jul 24 17:45:39.503395 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503355    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-token\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.503588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-envoy\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.503588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503444    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-podinfo\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.503588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7kwf\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-kube-api-access-t7kwf\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.503588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503512    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-credential-socket\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.503588 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503548    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istiod-ca-cert\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.503875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503601    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-certs\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.503875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-socket\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.503875 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.503694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-data\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605000 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.604957    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-envoy\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605000 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-podinfo\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7kwf\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-kube-api-access-t7kwf\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605083    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-credential-socket\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605129    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istiod-ca-cert\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605180    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-certs\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605266    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-socket\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605534 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605288    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-data\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605534 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605323    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-token\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605534 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605495    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-credential-socket\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-data\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.605941 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605920    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-certs\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.606039 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.605927    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-socket\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.606346 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.606326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istiod-ca-cert\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.607516 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.607497    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-envoy\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.608953 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.608933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-podinfo\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.613323 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.613299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-token\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.613417 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.613400    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7kwf\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-kube-api-access-t7kwf\") pod \"e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") " pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.702717 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.702692    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:39.880567 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.880533    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"]
Jul 24 17:45:39.884535 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:45:39.884504    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f6b44c_fb9e_412b_bd92_1710e83f43aa.slice/crio-16d64f98a69b0c0d5dfa569b0f600542c519fc2ef9ec005825af1c1d81766fd5 WatchSource:0}: Error finding container 16d64f98a69b0c0d5dfa569b0f600542c519fc2ef9ec005825af1c1d81766fd5: Status 404 returned error can't find the container with id 16d64f98a69b0c0d5dfa569b0f600542c519fc2ef9ec005825af1c1d81766fd5
Jul 24 17:45:39.887518 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.887485    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:39.887619 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.887565    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:39.887619 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:39.887612    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:40.411043 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:40.410986    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" event={"ID":"f0f6b44c-fb9e-412b-bd92-1710e83f43aa","Type":"ContainerStarted","Data":"c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f"}
Jul 24 17:45:40.411577 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:40.411071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" event={"ID":"f0f6b44c-fb9e-412b-bd92-1710e83f43aa","Type":"ContainerStarted","Data":"16d64f98a69b0c0d5dfa569b0f600542c519fc2ef9ec005825af1c1d81766fd5"}
Jul 24 17:45:40.435033 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:40.434966    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" podStartSLOduration=1.434949335 podStartE2EDuration="1.434949335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:40.433750222 +0000 UTC m=+2129.404882402" watchObservedRunningTime="2026-07-24 17:45:40.434949335 +0000 UTC m=+2129.406081515"
Jul 24 17:45:40.703781 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:40.703751    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:40.708729 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:40.708705    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:41.415058 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:41.415000    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:41.416115 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:41.416097    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:42.400932 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:42.400905    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:45:43.026212 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.026179    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"]
Jul 24 17:45:43.029412 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.029396    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:43.032840 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.032821    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-22bf5f\""
Jul 24 17:45:43.043104 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.043083    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"]
Jul 24 17:45:43.139111 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.139080    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1638143e-a546-47e8-a618-5ce50521c9dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc\" (UID: \"1638143e-a546-47e8-a618-5ce50521c9dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:43.139286 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.139124    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f6l2\" (UniqueName: \"kubernetes.io/projected/1638143e-a546-47e8-a618-5ce50521c9dd-kube-api-access-9f6l2\") pod \"payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc\" (UID: \"1638143e-a546-47e8-a618-5ce50521c9dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:43.240392 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.240354    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1638143e-a546-47e8-a618-5ce50521c9dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc\" (UID: \"1638143e-a546-47e8-a618-5ce50521c9dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:43.240578 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.240403    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f6l2\" (UniqueName: \"kubernetes.io/projected/1638143e-a546-47e8-a618-5ce50521c9dd-kube-api-access-9f6l2\") pod \"payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc\" (UID: \"1638143e-a546-47e8-a618-5ce50521c9dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:43.241102 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.241084    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1638143e-a546-47e8-a618-5ce50521c9dd-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc\" (UID: \"1638143e-a546-47e8-a618-5ce50521c9dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:43.248674 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.248652    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f6l2\" (UniqueName: \"kubernetes.io/projected/1638143e-a546-47e8-a618-5ce50521c9dd-kube-api-access-9f6l2\") pod \"payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc\" (UID: \"1638143e-a546-47e8-a618-5ce50521c9dd\") " pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:43.340621 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.340518    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:43.526119 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:43.526096    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"]
Jul 24 17:45:43.527941 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:45:43.527914    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1638143e_a546_47e8_a618_5ce50521c9dd.slice/crio-e137e911adb2e8fc7859b93721e600e358b5b6af41ff51e15a6bc73775782182 WatchSource:0}: Error finding container e137e911adb2e8fc7859b93721e600e358b5b6af41ff51e15a6bc73775782182: Status 404 returned error can't find the container with id e137e911adb2e8fc7859b93721e600e358b5b6af41ff51e15a6bc73775782182
Jul 24 17:45:44.431232 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:44.431194    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc" event={"ID":"1638143e-a546-47e8-a618-5ce50521c9dd","Type":"ContainerStarted","Data":"8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca"}
Jul 24 17:45:44.431637 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:44.431243    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc" event={"ID":"1638143e-a546-47e8-a618-5ce50521c9dd","Type":"ContainerStarted","Data":"e137e911adb2e8fc7859b93721e600e358b5b6af41ff51e15a6bc73775782182"}
Jul 24 17:45:44.431637 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:44.431330    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:44.446891 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:44.446845    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc" podStartSLOduration=1.44683191 podStartE2EDuration="1.44683191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:45:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:45:44.445803648 +0000 UTC m=+2133.416935829" watchObservedRunningTime="2026-07-24 17:45:44.44683191 +0000 UTC m=+2133.417964090"
Jul 24 17:45:47.397282 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:47.397238    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:45:47.397763 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:47.397553    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:45:48.372542 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:48.372497    2562 patch_prober.go:28] interesting pod/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:45:48.372826 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:48.372587    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 24 17:45:48.455452 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:48.455412    2562 generic.go:356] "Generic (PLEG): container finished" podID="581c826c-f536-4569-b2c3-c6156fbb000d" containerID="e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56" exitCode=0
Jul 24 17:45:48.455933 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:48.455474    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" event={"ID":"581c826c-f536-4569-b2c3-c6156fbb000d","Type":"ContainerDied","Data":"e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56"}
Jul 24 17:45:48.456033 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:48.455969    2562 scope.go:117] "RemoveContainer" containerID="e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56"
Jul 24 17:45:48.457282 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:48.457242    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:48.457386 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:48.457335    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:48.457451 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:48.457381    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:49.465702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:49.465656    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" event={"ID":"581c826c-f536-4569-b2c3-c6156fbb000d","Type":"ContainerStarted","Data":"588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8"}
Jul 24 17:45:50.212460 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:50.212426    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:50.212738 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:50.212710    2562 patch_prober.go:28] interesting pod/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:45:50.212811 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:50.212788    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 24 17:45:51.213496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:51.213462    2562 patch_prober.go:28] interesting pod/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:45:51.213960 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:51.213523    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 24 17:45:52.213162 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:52.213126    2562 patch_prober.go:28] interesting pod/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:45:52.213341 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:52.213179    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 24 17:45:53.213336 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:53.213298    2562 patch_prober.go:28] interesting pod/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 24 17:45:53.213723 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:53.213361    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 24 17:45:54.216656 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.216625    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:54.217122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.216866    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:54.217575 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.217561    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:45:54.499726 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.499636    2562 generic.go:356] "Generic (PLEG): container finished" podID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerID="c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f" exitCode=0
Jul 24 17:45:54.499905 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.499711    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" event={"ID":"f0f6b44c-fb9e-412b-bd92-1710e83f43aa","Type":"ContainerDied","Data":"c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f"}
Jul 24 17:45:54.500128 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.500111    2562 scope.go:117] "RemoveContainer" containerID="c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f"
Jul 24 17:45:54.501103 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.501078    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:54.501171 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.501137    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:54.501171 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:54.501166    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:45:55.439546 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:55.439520    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:45:55.506251 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:55.506215    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" event={"ID":"f0f6b44c-fb9e-412b-bd92-1710e83f43aa","Type":"ContainerStarted","Data":"8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a"}
Jul 24 17:45:55.703056 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:55.703001    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:55.704203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:55.704179    2562 patch_prober.go:28] interesting pod/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 24 17:45:55.704311 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:55.704238    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 24 17:45:56.415334 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:56.415278    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:56.703640 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:56.703613    2562 patch_prober.go:28] interesting pod/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 24 17:45:56.704066 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:56.703679    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 24 17:45:57.703416 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:57.703380    2562 patch_prober.go:28] interesting pod/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 24 17:45:57.703586 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:57.703442    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 24 17:45:58.703264 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:58.703231    2562 patch_prober.go:28] interesting pod/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 24 17:45:58.703662 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:58.703298    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 24 17:45:59.707238 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:59.707203    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:45:59.708117 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:45:59.708098    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:47:27.884938 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:27.884897    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"]
Jul 24 17:47:27.885497 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:27.885127    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x" podUID="cecc0e12-eea3-495e-b021-594a3d760693" containerName="maas-api" containerID="cri-o://7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296" gracePeriod=30
Jul 24 17:47:28.003607 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.003576    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"]
Jul 24 17:47:28.003817 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.003796    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs" podUID="e4dcba02-0d8c-4456-b3f1-2704d70b415d" containerName="payload-processing" containerID="cri-o://b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230" gracePeriod=30
Jul 24 17:47:28.013075 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.013046    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"]
Jul 24 17:47:28.013303 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.013275    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh" podUID="adb647dd-521c-469b-acb4-99154b8590a6" containerName="payload-pre-processing" containerID="cri-o://c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e" gracePeriod=30
Jul 24 17:47:28.320798 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.320768    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:47:28.322756 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.322739    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:47:28.341058 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.341029    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:47:28.383061 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383033    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adb647dd-521c-469b-acb4-99154b8590a6-plugins-config-volume\") pod \"adb647dd-521c-469b-acb4-99154b8590a6\" (UID: \"adb647dd-521c-469b-acb4-99154b8590a6\") "
Jul 24 17:47:28.383219 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383072    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtbt8\" (UniqueName: \"kubernetes.io/projected/e4dcba02-0d8c-4456-b3f1-2704d70b415d-kube-api-access-dtbt8\") pod \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\" (UID: \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\") "
Jul 24 17:47:28.383219 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383118    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls\") pod \"cecc0e12-eea3-495e-b021-594a3d760693\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") "
Jul 24 17:47:28.383219 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383188    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7csgn\" (UniqueName: \"kubernetes.io/projected/cecc0e12-eea3-495e-b021-594a3d760693-kube-api-access-7csgn\") pod \"cecc0e12-eea3-495e-b021-594a3d760693\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") "
Jul 24 17:47:28.383399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383226    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecc0e12-eea3-495e-b021-594a3d760693-openshift-service-ca\") pod \"cecc0e12-eea3-495e-b021-594a3d760693\" (UID: \"cecc0e12-eea3-495e-b021-594a3d760693\") "
Jul 24 17:47:28.383399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383261    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgkxw\" (UniqueName: \"kubernetes.io/projected/adb647dd-521c-469b-acb4-99154b8590a6-kube-api-access-pgkxw\") pod \"adb647dd-521c-469b-acb4-99154b8590a6\" (UID: \"adb647dd-521c-469b-acb4-99154b8590a6\") "
Jul 24 17:47:28.383399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383297    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4dcba02-0d8c-4456-b3f1-2704d70b415d-plugins-config-volume\") pod \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\" (UID: \"e4dcba02-0d8c-4456-b3f1-2704d70b415d\") "
Jul 24 17:47:28.383731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383417    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adb647dd-521c-469b-acb4-99154b8590a6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "adb647dd-521c-469b-acb4-99154b8590a6" (UID: "adb647dd-521c-469b-acb4-99154b8590a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:47:28.383731 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.383589    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cecc0e12-eea3-495e-b021-594a3d760693-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cecc0e12-eea3-495e-b021-594a3d760693" (UID: "cecc0e12-eea3-495e-b021-594a3d760693"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:47:28.384115 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.384085    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4dcba02-0d8c-4456-b3f1-2704d70b415d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e4dcba02-0d8c-4456-b3f1-2704d70b415d" (UID: "e4dcba02-0d8c-4456-b3f1-2704d70b415d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:47:28.385625 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.385595    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4dcba02-0d8c-4456-b3f1-2704d70b415d-kube-api-access-dtbt8" (OuterVolumeSpecName: "kube-api-access-dtbt8") pod "e4dcba02-0d8c-4456-b3f1-2704d70b415d" (UID: "e4dcba02-0d8c-4456-b3f1-2704d70b415d"). InnerVolumeSpecName "kube-api-access-dtbt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:28.385625 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.385601    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cecc0e12-eea3-495e-b021-594a3d760693-kube-api-access-7csgn" (OuterVolumeSpecName: "kube-api-access-7csgn") pod "cecc0e12-eea3-495e-b021-594a3d760693" (UID: "cecc0e12-eea3-495e-b021-594a3d760693"). InnerVolumeSpecName "kube-api-access-7csgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:28.385830 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.385806    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cecc0e12-eea3-495e-b021-594a3d760693" (UID: "cecc0e12-eea3-495e-b021-594a3d760693"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:47:28.385924 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.385838    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adb647dd-521c-469b-acb4-99154b8590a6-kube-api-access-pgkxw" (OuterVolumeSpecName: "kube-api-access-pgkxw") pod "adb647dd-521c-469b-acb4-99154b8590a6" (UID: "adb647dd-521c-469b-acb4-99154b8590a6"). InnerVolumeSpecName "kube-api-access-pgkxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:28.484718 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.484683    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cecc0e12-eea3-495e-b021-594a3d760693-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:28.484718 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.484718    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7csgn\" (UniqueName: \"kubernetes.io/projected/cecc0e12-eea3-495e-b021-594a3d760693-kube-api-access-7csgn\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:28.484718 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.484728    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cecc0e12-eea3-495e-b021-594a3d760693-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:28.484918 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.484737    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgkxw\" (UniqueName: \"kubernetes.io/projected/adb647dd-521c-469b-acb4-99154b8590a6-kube-api-access-pgkxw\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:28.484918 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.484746    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4dcba02-0d8c-4456-b3f1-2704d70b415d-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:28.484918 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.484754    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adb647dd-521c-469b-acb4-99154b8590a6-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:28.484918 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.484762    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtbt8\" (UniqueName: \"kubernetes.io/projected/e4dcba02-0d8c-4456-b3f1-2704d70b415d-kube-api-access-dtbt8\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:28.920449 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.920368    2562 generic.go:356] "Generic (PLEG): container finished" podID="adb647dd-521c-469b-acb4-99154b8590a6" containerID="c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e" exitCode=0
Jul 24 17:47:28.920449 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.920418    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"
Jul 24 17:47:28.920927 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.920453    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh" event={"ID":"adb647dd-521c-469b-acb4-99154b8590a6","Type":"ContainerDied","Data":"c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e"}
Jul 24 17:47:28.920927 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.920518    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh" event={"ID":"adb647dd-521c-469b-acb4-99154b8590a6","Type":"ContainerDied","Data":"d230224781f5c8afa607edb713fda6b38c8948ddca5f178602f089395079a8eb"}
Jul 24 17:47:28.920927 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.920548    2562 scope.go:117] "RemoveContainer" containerID="c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e"
Jul 24 17:47:28.921804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.921767    2562 generic.go:356] "Generic (PLEG): container finished" podID="cecc0e12-eea3-495e-b021-594a3d760693" containerID="7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296" exitCode=0
Jul 24 17:47:28.921864 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.921817    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"
Jul 24 17:47:28.921917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.921857    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x" event={"ID":"cecc0e12-eea3-495e-b021-594a3d760693","Type":"ContainerDied","Data":"7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296"}
Jul 24 17:47:28.921917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.921902    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x" event={"ID":"cecc0e12-eea3-495e-b021-594a3d760693","Type":"ContainerDied","Data":"b33c46685258763219bee589a91b566a72d3811d69fa0ce7040cb777143d0e8a"}
Jul 24 17:47:28.923293 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.923271    2562 generic.go:356] "Generic (PLEG): container finished" podID="e4dcba02-0d8c-4456-b3f1-2704d70b415d" containerID="b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230" exitCode=0
Jul 24 17:47:28.923404 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.923310    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs" event={"ID":"e4dcba02-0d8c-4456-b3f1-2704d70b415d","Type":"ContainerDied","Data":"b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230"}
Jul 24 17:47:28.923404 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.923323    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"
Jul 24 17:47:28.923404 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.923337    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs" event={"ID":"e4dcba02-0d8c-4456-b3f1-2704d70b415d","Type":"ContainerDied","Data":"44202bb0e225706a21265f39291c0d01b4e8082799f36db2f9960fab0c80fa40"}
Jul 24 17:47:28.938589 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.938374    2562 scope.go:117] "RemoveContainer" containerID="c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e"
Jul 24 17:47:28.938818 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:47:28.938790    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e\": container with ID starting with c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e not found: ID does not exist" containerID="c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e"
Jul 24 17:47:28.938969 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.938931    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e"} err="failed to get container status \"c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e\": rpc error: code = NotFound desc = could not find container \"c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e\": container with ID starting with c52f359e0c5463bbd9b13bd0ec6b79148bef4c284646df31586498e5faa1817e not found: ID does not exist"
Jul 24 17:47:28.938969 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.938966    2562 scope.go:117] "RemoveContainer" containerID="7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296"
Jul 24 17:47:28.947318 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.947298    2562 scope.go:117] "RemoveContainer" containerID="7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296"
Jul 24 17:47:28.947593 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:47:28.947576    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296\": container with ID starting with 7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296 not found: ID does not exist" containerID="7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296"
Jul 24 17:47:28.947655 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.947600    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296"} err="failed to get container status \"7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296\": rpc error: code = NotFound desc = could not find container \"7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296\": container with ID starting with 7a83edaed26218b482d0871487f93c1759d8a8aae56ada7495f05dd970e70296 not found: ID does not exist"
Jul 24 17:47:28.947655 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.947617    2562 scope.go:117] "RemoveContainer" containerID="b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230"
Jul 24 17:47:28.952554 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.952513    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"]
Jul 24 17:47:28.954979 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.954959    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-387e6e-6f4895b5b-x59rh"]
Jul 24 17:47:28.956072 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.956058    2562 scope.go:117] "RemoveContainer" containerID="b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230"
Jul 24 17:47:28.956371 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:47:28.956353    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230\": container with ID starting with b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230 not found: ID does not exist" containerID="b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230"
Jul 24 17:47:28.956429 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.956376    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230"} err="failed to get container status \"b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230\": rpc error: code = NotFound desc = could not find container \"b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230\": container with ID starting with b78f1a57bfae8e47defe556e5ab1109660dc6d5a6c30124f248fe826da90e230 not found: ID does not exist"
Jul 24 17:47:28.964688 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.964667    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"]
Jul 24 17:47:28.968945 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.968925    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-387e6e-67db49cf8-47sbs"]
Jul 24 17:47:28.980227 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.980209    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"]
Jul 24 17:47:28.984308 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:28.984277    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-387e6e-f98bdfdcb-76f9x"]
Jul 24 17:47:29.504532 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:29.504491    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adb647dd-521c-469b-acb4-99154b8590a6" path="/var/lib/kubelet/pods/adb647dd-521c-469b-acb4-99154b8590a6/volumes"
Jul 24 17:47:29.504889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:29.504875    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cecc0e12-eea3-495e-b021-594a3d760693" path="/var/lib/kubelet/pods/cecc0e12-eea3-495e-b021-594a3d760693/volumes"
Jul 24 17:47:29.505203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:29.505190    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4dcba02-0d8c-4456-b3f1-2704d70b415d" path="/var/lib/kubelet/pods/e4dcba02-0d8c-4456-b3f1-2704d70b415d/volumes"
Jul 24 17:47:39.303246 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:39.303171    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"]
Jul 24 17:47:39.303600 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:39.303428    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy" containerID="cri-o://588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8" gracePeriod=30
Jul 24 17:47:44.552908 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.552885    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:47:44.713653 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.713622    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-istio-token\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.713815 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.713665    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-envoy\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.713815 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.713708    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-data\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.713815 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.713733    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581c826c-f536-4569-b2c3-c6156fbb000d-istiod-ca-cert\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.713815 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.713761    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8wvn\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-kube-api-access-g8wvn\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.713815 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.713805    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-credential-socket\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.714121 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.713925    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-socket\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.714121 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.713968    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581c826c-f536-4569-b2c3-c6156fbb000d-istio-podinfo\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.714121 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.714003    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-certs\") pod \"581c826c-f536-4569-b2c3-c6156fbb000d\" (UID: \"581c826c-f536-4569-b2c3-c6156fbb000d\") "
Jul 24 17:47:44.714277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.714225    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:44.714277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.714247    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/581c826c-f536-4569-b2c3-c6156fbb000d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:47:44.714385 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.714318    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:44.714385 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.714363    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581c826c-f536-4569-b2c3-c6156fbb000d-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.714385 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.714381    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.714565 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.714445    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:44.714689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.714671    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-data" (OuterVolumeSpecName: "istio-data") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:44.716183 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.716161    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:47:44.716410 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.716389    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/581c826c-f536-4569-b2c3-c6156fbb000d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:47:44.716548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.716526    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-kube-api-access-g8wvn" (OuterVolumeSpecName: "kube-api-access-g8wvn") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "kube-api-access-g8wvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:44.716548 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.716534    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-istio-token" (OuterVolumeSpecName: "istio-token") pod "581c826c-f536-4569-b2c3-c6156fbb000d" (UID: "581c826c-f536-4569-b2c3-c6156fbb000d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:44.815402 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.815374    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.815402 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.815398    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.815402 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.815406    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.815593 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.815413    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.815593 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.815423    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8wvn\" (UniqueName: \"kubernetes.io/projected/581c826c-f536-4569-b2c3-c6156fbb000d-kube-api-access-g8wvn\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.815593 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.815432    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581c826c-f536-4569-b2c3-c6156fbb000d-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.815593 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.815440    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581c826c-f536-4569-b2c3-c6156fbb000d-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:44.996436 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.996345    2562 generic.go:356] "Generic (PLEG): container finished" podID="581c826c-f536-4569-b2c3-c6156fbb000d" containerID="588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8" exitCode=0
Jul 24 17:47:44.996436 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.996400    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"
Jul 24 17:47:44.996436 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.996400    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" event={"ID":"581c826c-f536-4569-b2c3-c6156fbb000d","Type":"ContainerDied","Data":"588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8"}
Jul 24 17:47:44.996654 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.996456    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2" event={"ID":"581c826c-f536-4569-b2c3-c6156fbb000d","Type":"ContainerDied","Data":"238bfafc863d712c805dc4bbb0bb451fe4d9ac091c419421655ad300dae59924"}
Jul 24 17:47:44.996654 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:44.996486    2562 scope.go:117] "RemoveContainer" containerID="588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8"
Jul 24 17:47:45.006095 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:45.006078    2562 scope.go:117] "RemoveContainer" containerID="e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56"
Jul 24 17:47:45.013765 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:45.013746    2562 scope.go:117] "RemoveContainer" containerID="588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8"
Jul 24 17:47:45.014038 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:47:45.014005    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8\": container with ID starting with 588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8 not found: ID does not exist" containerID="588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8"
Jul 24 17:47:45.014092 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:45.014047    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8"} err="failed to get container status \"588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8\": rpc error: code = NotFound desc = could not find container \"588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8\": container with ID starting with 588db7f4cae35dd814e3297c317329d3c2cd9619fac454037630826fe698c2c8 not found: ID does not exist"
Jul 24 17:47:45.014092 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:45.014073    2562 scope.go:117] "RemoveContainer" containerID="e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56"
Jul 24 17:47:45.014317 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:47:45.014300    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56\": container with ID starting with e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56 not found: ID does not exist" containerID="e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56"
Jul 24 17:47:45.014359 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:45.014324    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56"} err="failed to get container status \"e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56\": rpc error: code = NotFound desc = could not find container \"e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56\": container with ID starting with e6af9dbe762c74a5897da9a916b68c680a67758779723952d0595d884ba47a56 not found: ID does not exist"
Jul 24 17:47:45.017791 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:45.017773    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"]
Jul 24 17:47:45.021568 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:45.021548    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-387e6e-openshift-default-58857fc846-hf6g2"]
Jul 24 17:47:45.504355 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:45.504322    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" path="/var/lib/kubelet/pods/581c826c-f536-4569-b2c3-c6156fbb000d/volumes"
Jul 24 17:47:50.195570 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.195537    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"]
Jul 24 17:47:50.196175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.195758    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc" podUID="1638143e-a546-47e8-a618-5ce50521c9dd" containerName="payload-processing" containerID="cri-o://8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca" gracePeriod=30
Jul 24 17:47:50.444360 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.444336    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:47:50.565410 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.565320    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1638143e-a546-47e8-a618-5ce50521c9dd-plugins-config-volume\") pod \"1638143e-a546-47e8-a618-5ce50521c9dd\" (UID: \"1638143e-a546-47e8-a618-5ce50521c9dd\") "
Jul 24 17:47:50.565410 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.565358    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9f6l2\" (UniqueName: \"kubernetes.io/projected/1638143e-a546-47e8-a618-5ce50521c9dd-kube-api-access-9f6l2\") pod \"1638143e-a546-47e8-a618-5ce50521c9dd\" (UID: \"1638143e-a546-47e8-a618-5ce50521c9dd\") "
Jul 24 17:47:50.565741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.565705    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1638143e-a546-47e8-a618-5ce50521c9dd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1638143e-a546-47e8-a618-5ce50521c9dd" (UID: "1638143e-a546-47e8-a618-5ce50521c9dd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:47:50.567512 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.567490    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1638143e-a546-47e8-a618-5ce50521c9dd-kube-api-access-9f6l2" (OuterVolumeSpecName: "kube-api-access-9f6l2") pod "1638143e-a546-47e8-a618-5ce50521c9dd" (UID: "1638143e-a546-47e8-a618-5ce50521c9dd"). InnerVolumeSpecName "kube-api-access-9f6l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:47:50.666634 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.666610    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1638143e-a546-47e8-a618-5ce50521c9dd-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:50.666634 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:50.666631    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f6l2\" (UniqueName: \"kubernetes.io/projected/1638143e-a546-47e8-a618-5ce50521c9dd-kube-api-access-9f6l2\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:47:51.029214 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.029175    2562 generic.go:356] "Generic (PLEG): container finished" podID="1638143e-a546-47e8-a618-5ce50521c9dd" containerID="8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca" exitCode=0
Jul 24 17:47:51.029403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.029227    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"
Jul 24 17:47:51.029403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.029267    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc" event={"ID":"1638143e-a546-47e8-a618-5ce50521c9dd","Type":"ContainerDied","Data":"8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca"}
Jul 24 17:47:51.029403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.029320    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc" event={"ID":"1638143e-a546-47e8-a618-5ce50521c9dd","Type":"ContainerDied","Data":"e137e911adb2e8fc7859b93721e600e358b5b6af41ff51e15a6bc73775782182"}
Jul 24 17:47:51.029403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.029343    2562 scope.go:117] "RemoveContainer" containerID="8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca"
Jul 24 17:47:51.038041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.038005    2562 scope.go:117] "RemoveContainer" containerID="8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca"
Jul 24 17:47:51.038311 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:47:51.038286    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca\": container with ID starting with 8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca not found: ID does not exist" containerID="8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca"
Jul 24 17:47:51.038405 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.038320    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca"} err="failed to get container status \"8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca\": rpc error: code = NotFound desc = could not find container \"8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca\": container with ID starting with 8fd06bf9f1409d8ff4b8a584d5f6e529b1de64db6ac5e631f5c206c5f6fd5eca not found: ID does not exist"
Jul 24 17:47:51.066494 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.066473    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"]
Jul 24 17:47:51.070213 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.070190    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-22bf5f-d6b99694-4wphc"]
Jul 24 17:47:51.504623 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:47:51.504585    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1638143e-a546-47e8-a618-5ce50521c9dd" path="/var/lib/kubelet/pods/1638143e-a546-47e8-a618-5ce50521c9dd/volumes"
Jul 24 17:48:01.360322 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:01.360280    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"]
Jul 24 17:48:01.360791 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:01.360536    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy" containerID="cri-o://8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a" gracePeriod=30
Jul 24 17:48:02.236715 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.236683    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"]
Jul 24 17:48:02.237036 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237006    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adb647dd-521c-469b-acb4-99154b8590a6" containerName="payload-pre-processing"
Jul 24 17:48:02.237096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237038    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="adb647dd-521c-469b-acb4-99154b8590a6" containerName="payload-pre-processing"
Jul 24 17:48:02.237096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237055    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4dcba02-0d8c-4456-b3f1-2704d70b415d" containerName="payload-processing"
Jul 24 17:48:02.237096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237062    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4dcba02-0d8c-4456-b3f1-2704d70b415d" containerName="payload-processing"
Jul 24 17:48:02.237096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237070    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1638143e-a546-47e8-a618-5ce50521c9dd" containerName="payload-processing"
Jul 24 17:48:02.237096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237076    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1638143e-a546-47e8-a618-5ce50521c9dd" containerName="payload-processing"
Jul 24 17:48:02.237096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237085    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy"
Jul 24 17:48:02.237096 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237091    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237099    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cecc0e12-eea3-495e-b021-594a3d760693" containerName="maas-api"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237105    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cecc0e12-eea3-495e-b021-594a3d760693" containerName="maas-api"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237113    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237118    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237161    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="adb647dd-521c-469b-acb4-99154b8590a6" containerName="payload-pre-processing"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237169    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4dcba02-0d8c-4456-b3f1-2704d70b415d" containerName="payload-processing"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237177    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cecc0e12-eea3-495e-b021-594a3d760693" containerName="maas-api"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237183    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237188    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1638143e-a546-47e8-a618-5ce50521c9dd" containerName="payload-processing"
Jul 24 17:48:02.237309 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.237195    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="581c826c-f536-4569-b2c3-c6156fbb000d" containerName="istio-proxy"
Jul 24 17:48:02.241366 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.241347    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.244230 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.244214    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-cf30f0-openshift-default-dockercfg-2tdfz\""
Jul 24 17:48:02.252864 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.252841    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"]
Jul 24 17:48:02.361671 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361638    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-certs\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.362144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-socket\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.362144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-envoy\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.362144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-data\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.362144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istiod-ca-cert\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.362144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-credential-socket\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.362144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361830    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpqzm\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-kube-api-access-fpqzm\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.362144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-token\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.362144 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.361875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-podinfo\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-socket\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-envoy\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-data\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istiod-ca-cert\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-credential-socket\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463282    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpqzm\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-kube-api-access-fpqzm\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463320    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-token\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463350    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-podinfo\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463428 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463375    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-certs\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463794 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463595    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-socket\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463794 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-data\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463794 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463670    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-credential-socket\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.463917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.463849    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-certs\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.464257 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.464240    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istiod-ca-cert\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.465882 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.465856    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-envoy\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.465988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.465977    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-podinfo\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.471672 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.471654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpqzm\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-kube-api-access-fpqzm\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.471767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.471712    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-token\") pod \"e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") " pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.551506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.551430    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:02.683369 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.683346    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"]
Jul 24 17:48:02.686047 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:48:02.685993    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ebae9b_f44c_4e9f_85f8_c3ddd339c765.slice/crio-302fae74a962ea87afeacbf8fcc4371b495eaa7160412d1c21034328187ef46e WatchSource:0}: Error finding container 302fae74a962ea87afeacbf8fcc4371b495eaa7160412d1c21034328187ef46e: Status 404 returned error can't find the container with id 302fae74a962ea87afeacbf8fcc4371b495eaa7160412d1c21034328187ef46e
Jul 24 17:48:02.688720 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.688682    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:02.688805 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.688763    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:02.688845 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:02.688801    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:03.082771 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:03.082734    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" event={"ID":"50ebae9b-f44c-4e9f-85f8-c3ddd339c765","Type":"ContainerStarted","Data":"0296cd522f2e71cd3a974e87e2d396bf568e4c8b5be9daf3408dc0bedad30ee9"}
Jul 24 17:48:03.082935 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:03.082780    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" event={"ID":"50ebae9b-f44c-4e9f-85f8-c3ddd339c765","Type":"ContainerStarted","Data":"302fae74a962ea87afeacbf8fcc4371b495eaa7160412d1c21034328187ef46e"}
Jul 24 17:48:03.106998 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:03.106944    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" podStartSLOduration=1.106924813 podStartE2EDuration="1.106924813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:03.106701584 +0000 UTC m=+2272.077833764" watchObservedRunningTime="2026-07-24 17:48:03.106924813 +0000 UTC m=+2272.078056995"
Jul 24 17:48:03.551705 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:03.551677    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:03.556474 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:03.556450    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:04.087569 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:04.087530    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:04.088446 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:04.088427    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:05.095218 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.095181    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"]
Jul 24 17:48:05.098643 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.098624    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.103371 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.103347    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:48:05.103480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.103347    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:48:05.103480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.103354    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:48:05.103480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.103359    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-cf30f0-serving-cert\""
Jul 24 17:48:05.117223 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.117199    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"]
Jul 24 17:48:05.189473 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.189447    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46c686c6-ee7b-47f2-933f-a92003727ca6-openshift-service-ca\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.189621 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.189580    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tc52\" (UniqueName: \"kubernetes.io/projected/46c686c6-ee7b-47f2-933f-a92003727ca6-kube-api-access-7tc52\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.189712 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.189649    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46c686c6-ee7b-47f2-933f-a92003727ca6-maas-api-tls\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.290651 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.290621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tc52\" (UniqueName: \"kubernetes.io/projected/46c686c6-ee7b-47f2-933f-a92003727ca6-kube-api-access-7tc52\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.290823 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.290667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46c686c6-ee7b-47f2-933f-a92003727ca6-maas-api-tls\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.290823 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.290696    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46c686c6-ee7b-47f2-933f-a92003727ca6-openshift-service-ca\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.291302 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.291284    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46c686c6-ee7b-47f2-933f-a92003727ca6-openshift-service-ca\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.293269 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.293242    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46c686c6-ee7b-47f2-933f-a92003727ca6-maas-api-tls\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.301702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.301682    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tc52\" (UniqueName: \"kubernetes.io/projected/46c686c6-ee7b-47f2-933f-a92003727ca6-kube-api-access-7tc52\") pod \"maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.409453 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.409373    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:05.540064 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.540038    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"]
Jul 24 17:48:05.542011 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:48:05.541980    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c686c6_ee7b_47f2_933f_a92003727ca6.slice/crio-f69b32c8653c08c99319c04fe1298b6ff57a420f4b0f040464e2e15bf96c58b3 WatchSource:0}: Error finding container f69b32c8653c08c99319c04fe1298b6ff57a420f4b0f040464e2e15bf96c58b3: Status 404 returned error can't find the container with id f69b32c8653c08c99319c04fe1298b6ff57a420f4b0f040464e2e15bf96c58b3
Jul 24 17:48:05.543851 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.543832    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:48:05.815227 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.815196    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"]
Jul 24 17:48:05.819634 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.819618    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:05.823110 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.823091    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-cf30f0\""
Jul 24 17:48:05.840304 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.840275    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"]
Jul 24 17:48:05.844138 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.844116    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"]
Jul 24 17:48:05.847406 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.847389    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:05.851774 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.851758    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-cf30f0-dockercfg-dfvmb\""
Jul 24 17:48:05.867774 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.867756    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"]
Jul 24 17:48:05.895554 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.895530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc\" (UID: \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:05.895667 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.895556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49053ac-0a32-49a2-8de2-331312de64c6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h\" (UID: \"a49053ac-0a32-49a2-8de2-331312de64c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:05.895667 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.895609    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7n92\" (UniqueName: \"kubernetes.io/projected/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-kube-api-access-q7n92\") pod \"payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc\" (UID: \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:05.895667 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.895650    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn4md\" (UniqueName: \"kubernetes.io/projected/a49053ac-0a32-49a2-8de2-331312de64c6-kube-api-access-jn4md\") pod \"payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h\" (UID: \"a49053ac-0a32-49a2-8de2-331312de64c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:05.997090 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.997051    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn4md\" (UniqueName: \"kubernetes.io/projected/a49053ac-0a32-49a2-8de2-331312de64c6-kube-api-access-jn4md\") pod \"payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h\" (UID: \"a49053ac-0a32-49a2-8de2-331312de64c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:05.997263 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.997124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc\" (UID: \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:05.997263 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.997144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49053ac-0a32-49a2-8de2-331312de64c6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h\" (UID: \"a49053ac-0a32-49a2-8de2-331312de64c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:05.997263 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.997222    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7n92\" (UniqueName: \"kubernetes.io/projected/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-kube-api-access-q7n92\") pod \"payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc\" (UID: \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:05.997811 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.997793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc\" (UID: \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:05.997895 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:05.997879    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49053ac-0a32-49a2-8de2-331312de64c6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h\" (UID: \"a49053ac-0a32-49a2-8de2-331312de64c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:06.009195 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.009170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn4md\" (UniqueName: \"kubernetes.io/projected/a49053ac-0a32-49a2-8de2-331312de64c6-kube-api-access-jn4md\") pod \"payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h\" (UID: \"a49053ac-0a32-49a2-8de2-331312de64c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:06.010049 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.010007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7n92\" (UniqueName: \"kubernetes.io/projected/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-kube-api-access-q7n92\") pod \"payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc\" (UID: \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:06.100618 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.100589    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb" event={"ID":"46c686c6-ee7b-47f2-933f-a92003727ca6","Type":"ContainerStarted","Data":"f69b32c8653c08c99319c04fe1298b6ff57a420f4b0f040464e2e15bf96c58b3"}
Jul 24 17:48:06.130265 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.130244    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:06.156663 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.156624    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:06.280534 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.280141    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"]
Jul 24 17:48:06.282169 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:48:06.282128    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7287be2_5f49_4ba8_8a0b_91f5d5e36f2e.slice/crio-28705adb2e7bbc072a4597bf22958db9ef375a7f3fbb00c343757129dd8a159b WatchSource:0}: Error finding container 28705adb2e7bbc072a4597bf22958db9ef375a7f3fbb00c343757129dd8a159b: Status 404 returned error can't find the container with id 28705adb2e7bbc072a4597bf22958db9ef375a7f3fbb00c343757129dd8a159b
Jul 24 17:48:06.309047 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.309005    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"]
Jul 24 17:48:06.312233 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:48:06.312208    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49053ac_0a32_49a2_8de2_331312de64c6.slice/crio-3ba4c3d5045a140f7efb25e6d82a9f90ff10c12ee03cf5cd572e3c61bb343e2a WatchSource:0}: Error finding container 3ba4c3d5045a140f7efb25e6d82a9f90ff10c12ee03cf5cd572e3c61bb343e2a: Status 404 returned error can't find the container with id 3ba4c3d5045a140f7efb25e6d82a9f90ff10c12ee03cf5cd572e3c61bb343e2a
Jul 24 17:48:06.606510 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.606488    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:48:06.703652 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703622    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-token\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.703842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703665    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istiod-ca-cert\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.703842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703682    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-socket\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.703842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703700    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-podinfo\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.703842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703744    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-data\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.703842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703789    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-credential-socket\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.703842 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703822    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-certs\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.704203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7kwf\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-kube-api-access-t7kwf\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.704203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703886    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-envoy\") pod \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\" (UID: \"f0f6b44c-fb9e-412b-bd92-1710e83f43aa\") "
Jul 24 17:48:06.704203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.703983    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:06.704203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.704154    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:06.704203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.704184    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:06.704203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.704189    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:06.704413 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.704210    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:06.704719 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.704695    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:06.706131 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.706104    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:48:06.706248 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.706181    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:06.706248 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.706198    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-kube-api-access-t7kwf" (OuterVolumeSpecName: "kube-api-access-t7kwf") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "kube-api-access-t7kwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:06.706639 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.706619    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0f6b44c-fb9e-412b-bd92-1710e83f43aa" (UID: "f0f6b44c-fb9e-412b-bd92-1710e83f43aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:06.804792 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.804767    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:06.804792 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.804791    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7kwf\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-kube-api-access-t7kwf\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:06.804940 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.804800    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:06.804940 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.804810    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:06.804940 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.804817    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:06.804940 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.804824    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:06.804940 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.804833    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:06.804940 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:06.804840    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0f6b44c-fb9e-412b-bd92-1710e83f43aa-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:07.106917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.106884    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc" event={"ID":"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e","Type":"ContainerStarted","Data":"4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2"}
Jul 24 17:48:07.107399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.106977    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:07.107399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.107001    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc" event={"ID":"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e","Type":"ContainerStarted","Data":"28705adb2e7bbc072a4597bf22958db9ef375a7f3fbb00c343757129dd8a159b"}
Jul 24 17:48:07.108493 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.108465    2562 generic.go:356] "Generic (PLEG): container finished" podID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerID="8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a" exitCode=0
Jul 24 17:48:07.108716 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.108527    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"
Jul 24 17:48:07.108716 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.108560    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" event={"ID":"f0f6b44c-fb9e-412b-bd92-1710e83f43aa","Type":"ContainerDied","Data":"8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a"}
Jul 24 17:48:07.108716 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.108614    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh" event={"ID":"f0f6b44c-fb9e-412b-bd92-1710e83f43aa","Type":"ContainerDied","Data":"16d64f98a69b0c0d5dfa569b0f600542c519fc2ef9ec005825af1c1d81766fd5"}
Jul 24 17:48:07.108716 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.108645    2562 scope.go:117] "RemoveContainer" containerID="8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a"
Jul 24 17:48:07.110319 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.110197    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h" event={"ID":"a49053ac-0a32-49a2-8de2-331312de64c6","Type":"ContainerStarted","Data":"4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477"}
Jul 24 17:48:07.110433 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.110348    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:07.110433 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.110373    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h" event={"ID":"a49053ac-0a32-49a2-8de2-331312de64c6","Type":"ContainerStarted","Data":"3ba4c3d5045a140f7efb25e6d82a9f90ff10c12ee03cf5cd572e3c61bb343e2a"}
Jul 24 17:48:07.111740 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.111718    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb" event={"ID":"46c686c6-ee7b-47f2-933f-a92003727ca6","Type":"ContainerStarted","Data":"7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347"}
Jul 24 17:48:07.111880 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.111861    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:07.117590 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.117575    2562 scope.go:117] "RemoveContainer" containerID="c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f"
Jul 24 17:48:07.123381 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.123340    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc" podStartSLOduration=2.123326841 podStartE2EDuration="2.123326841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:07.122870286 +0000 UTC m=+2276.094002468" watchObservedRunningTime="2026-07-24 17:48:07.123326841 +0000 UTC m=+2276.094459029"
Jul 24 17:48:07.125575 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.125561    2562 scope.go:117] "RemoveContainer" containerID="8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a"
Jul 24 17:48:07.125808 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:48:07.125789    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a\": container with ID starting with 8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a not found: ID does not exist" containerID="8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a"
Jul 24 17:48:07.125852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.125816    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a"} err="failed to get container status \"8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a\": rpc error: code = NotFound desc = could not find container \"8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a\": container with ID starting with 8534322598ca2c4fbfdaf1e66253ddf01f140e3a1a75ad0320539c423e53403a not found: ID does not exist"
Jul 24 17:48:07.125852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.125836    2562 scope.go:117] "RemoveContainer" containerID="c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f"
Jul 24 17:48:07.126057 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:48:07.126039    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f\": container with ID starting with c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f not found: ID does not exist" containerID="c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f"
Jul 24 17:48:07.126109 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.126063    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f"} err="failed to get container status \"c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f\": rpc error: code = NotFound desc = could not find container \"c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f\": container with ID starting with c173e2948a8cc96c8f43b29c7ddfa176cafbff94418ade782df8b008bd84526f not found: ID does not exist"
Jul 24 17:48:07.141532 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.141497    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb" podStartSLOduration=1.604950697 podStartE2EDuration="2.141485552s" totalImagesPullingTime="536.534855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:07.140576295 +0000 UTC m=+2276.111708477" watchObservedRunningTime="2026-07-24 17:48:07.141485552 +0000 UTC m=+2276.112617754"
Jul 24 17:48:07.157192 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.157151    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h" podStartSLOduration=2.157135748 podStartE2EDuration="2.157135748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:07.156076089 +0000 UTC m=+2276.127208266" watchObservedRunningTime="2026-07-24 17:48:07.157135748 +0000 UTC m=+2276.128267932"
Jul 24 17:48:07.169717 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.169695    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"]
Jul 24 17:48:07.173920 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.173899    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-22bf5f-openshift-default-9ddc6dc94-z7fjh"]
Jul 24 17:48:07.505832 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:07.505803    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" path="/var/lib/kubelet/pods/f0f6b44c-fb9e-412b-bd92-1710e83f43aa/volumes"
Jul 24 17:48:10.264867 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.264838    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"]
Jul 24 17:48:10.265269 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.265211    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy"
Jul 24 17:48:10.265269 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.265223    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy"
Jul 24 17:48:10.265337 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.265288    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy"
Jul 24 17:48:10.265337 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.265297    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy"
Jul 24 17:48:10.269721 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.269704    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.272341 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.272318    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0e9e3f-openshift-default-dockercfg-k7c2l\""
Jul 24 17:48:10.297476 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.297455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"]
Jul 24 17:48:10.330489 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-podinfo\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.330596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w599\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-kube-api-access-9w599\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.330596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-certs\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.330596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330557    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-envoy\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.330596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-data\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.330596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330592    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-credential-socket\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.330811 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330641    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-token\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.330811 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330688    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istiod-ca-cert\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.330811 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.330734    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-socket\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.431974 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.431912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-socket\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432187 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.431986    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-podinfo\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432187 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432048    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w599\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-kube-api-access-9w599\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432187 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-certs\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432187 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432165    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-envoy\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432406 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432187    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-data\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432406 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-credential-socket\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432406 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-token\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432406 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istiod-ca-cert\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432627 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432417    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-socket\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432627 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432538    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-certs\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432734 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432637    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-credential-socket\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432734 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-data\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.432983 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.432962    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istiod-ca-cert\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.434657 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.434625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-envoy\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.434813 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.434795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-podinfo\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.442673 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.442649    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-token\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.444567 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.444547    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w599\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-kube-api-access-9w599\") pod \"e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") " pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.582480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.582412    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:10.743627 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.743598    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"]
Jul 24 17:48:10.744607 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:48:10.744582    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20bd451_2b2b_41b2_8fe3_f2b58ee8e80d.slice/crio-5d05f7aa615aea306f0734a3f1ffd98223394ebf17c4356be7b186bff34d72ac WatchSource:0}: Error finding container 5d05f7aa615aea306f0734a3f1ffd98223394ebf17c4356be7b186bff34d72ac: Status 404 returned error can't find the container with id 5d05f7aa615aea306f0734a3f1ffd98223394ebf17c4356be7b186bff34d72ac
Jul 24 17:48:10.746661 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.746628    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:10.746746 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.746691    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:10.746746 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:10.746720    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:11.134282 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:11.134244    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" event={"ID":"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d","Type":"ContainerStarted","Data":"9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8"}
Jul 24 17:48:11.134462 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:11.134307    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" event={"ID":"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d","Type":"ContainerStarted","Data":"5d05f7aa615aea306f0734a3f1ffd98223394ebf17c4356be7b186bff34d72ac"}
Jul 24 17:48:11.161170 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:11.161123    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" podStartSLOduration=1.161108354 podStartE2EDuration="1.161108354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:11.159760618 +0000 UTC m=+2280.130892801" watchObservedRunningTime="2026-07-24 17:48:11.161108354 +0000 UTC m=+2280.132240535"
Jul 24 17:48:11.583004 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:11.582969    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:11.587829 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:11.587808    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:12.138161 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:12.138129    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:12.139263 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:12.139240    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:13.123207 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.123176    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:13.947737 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.947698    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"]
Jul 24 17:48:13.948100 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.948083    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy"
Jul 24 17:48:13.948185 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.948103    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0f6b44c-fb9e-412b-bd92-1710e83f43aa" containerName="istio-proxy"
Jul 24 17:48:13.964474 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.964437    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:13.968130 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.968090    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"]
Jul 24 17:48:13.968628 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.968605    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0e9e3f-dockercfg-7nlh9\""
Jul 24 17:48:13.968965 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.968685    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0e9e3f\""
Jul 24 17:48:13.978471 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.978448    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"]
Jul 24 17:48:13.982740 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.982714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:13.992694 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:13.992666    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"]
Jul 24 17:48:14.066699 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.066659    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77df28a0-4c9d-4180-ac80-18ef38e4f919-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb\" (UID: \"77df28a0-4c9d-4180-ac80-18ef38e4f919\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:14.066996 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.066974    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08757ad-2b30-4e54-b53c-03f4db7b9507-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr\" (UID: \"d08757ad-2b30-4e54-b53c-03f4db7b9507\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:14.067252 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.067232    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff7w8\" (UniqueName: \"kubernetes.io/projected/d08757ad-2b30-4e54-b53c-03f4db7b9507-kube-api-access-ff7w8\") pod \"payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr\" (UID: \"d08757ad-2b30-4e54-b53c-03f4db7b9507\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:14.067398 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.067383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwkrl\" (UniqueName: \"kubernetes.io/projected/77df28a0-4c9d-4180-ac80-18ef38e4f919-kube-api-access-hwkrl\") pod \"payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb\" (UID: \"77df28a0-4c9d-4180-ac80-18ef38e4f919\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:14.168348 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.168302    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77df28a0-4c9d-4180-ac80-18ef38e4f919-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb\" (UID: \"77df28a0-4c9d-4180-ac80-18ef38e4f919\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:14.168852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.168356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08757ad-2b30-4e54-b53c-03f4db7b9507-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr\" (UID: \"d08757ad-2b30-4e54-b53c-03f4db7b9507\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:14.168852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.168442    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff7w8\" (UniqueName: \"kubernetes.io/projected/d08757ad-2b30-4e54-b53c-03f4db7b9507-kube-api-access-ff7w8\") pod \"payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr\" (UID: \"d08757ad-2b30-4e54-b53c-03f4db7b9507\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:14.168852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.168469    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwkrl\" (UniqueName: \"kubernetes.io/projected/77df28a0-4c9d-4180-ac80-18ef38e4f919-kube-api-access-hwkrl\") pod \"payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb\" (UID: \"77df28a0-4c9d-4180-ac80-18ef38e4f919\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:14.169757 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.169726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77df28a0-4c9d-4180-ac80-18ef38e4f919-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb\" (UID: \"77df28a0-4c9d-4180-ac80-18ef38e4f919\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:14.170504 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.170476    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08757ad-2b30-4e54-b53c-03f4db7b9507-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr\" (UID: \"d08757ad-2b30-4e54-b53c-03f4db7b9507\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:14.180061 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.179011    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff7w8\" (UniqueName: \"kubernetes.io/projected/d08757ad-2b30-4e54-b53c-03f4db7b9507-kube-api-access-ff7w8\") pod \"payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr\" (UID: \"d08757ad-2b30-4e54-b53c-03f4db7b9507\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:14.180061 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.179102    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwkrl\" (UniqueName: \"kubernetes.io/projected/77df28a0-4c9d-4180-ac80-18ef38e4f919-kube-api-access-hwkrl\") pod \"payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb\" (UID: \"77df28a0-4c9d-4180-ac80-18ef38e4f919\") " pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:14.282954 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.282858    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:14.294265 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.294232    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:14.461142 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.460653    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"]
Jul 24 17:48:14.482700 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:48:14.482668    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08757ad_2b30_4e54_b53c_03f4db7b9507.slice/crio-5a4407596e7f3f2179fca1a42e9cea16553e6ed4f4f995ed1b5c4828fd8333aa WatchSource:0}: Error finding container 5a4407596e7f3f2179fca1a42e9cea16553e6ed4f4f995ed1b5c4828fd8333aa: Status 404 returned error can't find the container with id 5a4407596e7f3f2179fca1a42e9cea16553e6ed4f4f995ed1b5c4828fd8333aa
Jul 24 17:48:14.702656 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:14.702623    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"]
Jul 24 17:48:14.704300 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:48:14.704272    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77df28a0_4c9d_4180_ac80_18ef38e4f919.slice/crio-c55db154564f3480bd21faa34aa707739d9aeb9e88575547c2a12d6b2383c03e WatchSource:0}: Error finding container c55db154564f3480bd21faa34aa707739d9aeb9e88575547c2a12d6b2383c03e: Status 404 returned error can't find the container with id c55db154564f3480bd21faa34aa707739d9aeb9e88575547c2a12d6b2383c03e
Jul 24 17:48:15.158804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:15.157845    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb" event={"ID":"77df28a0-4c9d-4180-ac80-18ef38e4f919","Type":"ContainerStarted","Data":"8fdd554fd938f533765d07fcdf7f74b71c0b25c065d366e9f69c702e97bf2f1e"}
Jul 24 17:48:15.158804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:15.157910    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb" event={"ID":"77df28a0-4c9d-4180-ac80-18ef38e4f919","Type":"ContainerStarted","Data":"c55db154564f3480bd21faa34aa707739d9aeb9e88575547c2a12d6b2383c03e"}
Jul 24 17:48:15.158804 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:15.158679    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:15.162290 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:15.160824    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr" event={"ID":"d08757ad-2b30-4e54-b53c-03f4db7b9507","Type":"ContainerStarted","Data":"74b221057773b6851e45ba888300fd542aa2e9cd9df8402b53869901f9f4f87c"}
Jul 24 17:48:15.162290 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:15.160870    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr" event={"ID":"d08757ad-2b30-4e54-b53c-03f4db7b9507","Type":"ContainerStarted","Data":"5a4407596e7f3f2179fca1a42e9cea16553e6ed4f4f995ed1b5c4828fd8333aa"}
Jul 24 17:48:15.162290 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:15.161395    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:15.197101 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:15.196991    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb" podStartSLOduration=2.19697537 podStartE2EDuration="2.19697537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:15.178519874 +0000 UTC m=+2284.149652056" watchObservedRunningTime="2026-07-24 17:48:15.19697537 +0000 UTC m=+2284.168107615"
Jul 24 17:48:15.198372 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:15.198341    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr" podStartSLOduration=2.198329279 podStartE2EDuration="2.198329279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:15.196011614 +0000 UTC m=+2284.167143794" watchObservedRunningTime="2026-07-24 17:48:15.198329279 +0000 UTC m=+2284.169461450"
Jul 24 17:48:18.119729 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:18.119651    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:18.120119 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:18.119738    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:20.130177 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:20.130144    2562 patch_prober.go:28] interesting pod/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 17:48:20.130542 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:20.130211    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 17:48:21.193902 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:21.193867    2562 generic.go:356] "Generic (PLEG): container finished" podID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerID="0296cd522f2e71cd3a974e87e2d396bf568e4c8b5be9daf3408dc0bedad30ee9" exitCode=0
Jul 24 17:48:21.194403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:21.193939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" event={"ID":"50ebae9b-f44c-4e9f-85f8-c3ddd339c765","Type":"ContainerDied","Data":"0296cd522f2e71cd3a974e87e2d396bf568e4c8b5be9daf3408dc0bedad30ee9"}
Jul 24 17:48:21.194403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:21.194303    2562 scope.go:117] "RemoveContainer" containerID="0296cd522f2e71cd3a974e87e2d396bf568e4c8b5be9daf3408dc0bedad30ee9"
Jul 24 17:48:21.200492 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:21.200456    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:21.200620 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:21.200553    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:21.200620 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:21.200602    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:22.200564 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:22.200530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" event={"ID":"50ebae9b-f44c-4e9f-85f8-c3ddd339c765","Type":"ContainerStarted","Data":"2c202a3e47d5ef8bce8bea0eaf921cb5787c34d34ee8d886bf0ce68a488dd258"}
Jul 24 17:48:22.552281 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:22.552194    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:22.552532 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:22.552425    2562 patch_prober.go:28] interesting pod/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 24 17:48:22.552695 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:22.552672    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 24 17:48:23.551958 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:23.551910    2562 patch_prober.go:28] interesting pod/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 24 17:48:23.552494 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:23.551980    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 24 17:48:24.552278 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:24.552234    2562 patch_prober.go:28] interesting pod/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 24 17:48:24.552809 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:24.552298    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 24 17:48:25.557044 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:25.553375    2562 patch_prober.go:28] interesting pod/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 24 17:48:25.557044 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:25.553434    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 24 17:48:26.554999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:26.554970    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:26.555271 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:26.555254    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:26.555958 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:26.555936    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:27.176102 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:27.176073    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:48:27.176569 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:27.176146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:48:28.180140 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:28.180108    2562 patch_prober.go:28] interesting pod/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 17:48:28.180552 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:28.180162    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 17:48:29.238733 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:29.238699    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerID="9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8" exitCode=0
Jul 24 17:48:29.239219 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:29.238777    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" event={"ID":"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d","Type":"ContainerDied","Data":"9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8"}
Jul 24 17:48:29.239284 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:29.239252    2562 scope.go:117] "RemoveContainer" containerID="9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8"
Jul 24 17:48:29.240358 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:29.240328    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:29.240482 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:29.240408    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:29.240482 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:29.240455    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:48:30.244780 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:30.244748    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" event={"ID":"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d","Type":"ContainerStarted","Data":"e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c"}
Jul 24 17:48:30.583658 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:30.583558    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:30.583846 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:30.583824    2562 patch_prober.go:28] interesting pod/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 24 17:48:30.583889 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:30.583872    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 24 17:48:31.583320 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:31.583281    2562 patch_prober.go:28] interesting pod/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 24 17:48:31.583719 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:31.583335    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 24 17:48:32.583295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:32.583258    2562 patch_prober.go:28] interesting pod/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 24 17:48:32.583475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:32.583313    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 24 17:48:33.582794 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:33.582760    2562 patch_prober.go:28] interesting pod/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 24 17:48:33.582980 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:33.582806    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 24 17:48:34.586685 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:34.586655    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:34.587086 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:34.586932    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:34.587568 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:34.587553    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:48:40.702686 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:40.702652    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"]
Jul 24 17:48:40.703162 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:40.702862    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb" podUID="46c686c6-ee7b-47f2-933f-a92003727ca6" containerName="maas-api" containerID="cri-o://7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347" gracePeriod=30
Jul 24 17:48:40.758616 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:40.758589    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"]
Jul 24 17:48:40.758810 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:40.758784    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h" podUID="a49053ac-0a32-49a2-8de2-331312de64c6" containerName="payload-processing" containerID="cri-o://4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477" gracePeriod=30
Jul 24 17:48:40.763435 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:40.763393    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"]
Jul 24 17:48:40.763616 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:40.763596    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc" podUID="c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e" containerName="payload-pre-processing" containerID="cri-o://4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2" gracePeriod=30
Jul 24 17:48:41.034723 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.034700    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:41.037796 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.037774    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:41.059215 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.059196    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:41.222084 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222053    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn4md\" (UniqueName: \"kubernetes.io/projected/a49053ac-0a32-49a2-8de2-331312de64c6-kube-api-access-jn4md\") pod \"a49053ac-0a32-49a2-8de2-331312de64c6\" (UID: \"a49053ac-0a32-49a2-8de2-331312de64c6\") "
Jul 24 17:48:41.222273 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222105    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7n92\" (UniqueName: \"kubernetes.io/projected/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-kube-api-access-q7n92\") pod \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\" (UID: \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\") "
Jul 24 17:48:41.222273 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222144    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46c686c6-ee7b-47f2-933f-a92003727ca6-maas-api-tls\") pod \"46c686c6-ee7b-47f2-933f-a92003727ca6\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") "
Jul 24 17:48:41.222273 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222194    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tc52\" (UniqueName: \"kubernetes.io/projected/46c686c6-ee7b-47f2-933f-a92003727ca6-kube-api-access-7tc52\") pod \"46c686c6-ee7b-47f2-933f-a92003727ca6\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") "
Jul 24 17:48:41.222273 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222218    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49053ac-0a32-49a2-8de2-331312de64c6-plugins-config-volume\") pod \"a49053ac-0a32-49a2-8de2-331312de64c6\" (UID: \"a49053ac-0a32-49a2-8de2-331312de64c6\") "
Jul 24 17:48:41.222489 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222280    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46c686c6-ee7b-47f2-933f-a92003727ca6-openshift-service-ca\") pod \"46c686c6-ee7b-47f2-933f-a92003727ca6\" (UID: \"46c686c6-ee7b-47f2-933f-a92003727ca6\") "
Jul 24 17:48:41.222489 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222315    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-plugins-config-volume\") pod \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\" (UID: \"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e\") "
Jul 24 17:48:41.222657 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222624    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46c686c6-ee7b-47f2-933f-a92003727ca6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46c686c6-ee7b-47f2-933f-a92003727ca6" (UID: "46c686c6-ee7b-47f2-933f-a92003727ca6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:41.222723 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a49053ac-0a32-49a2-8de2-331312de64c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a49053ac-0a32-49a2-8de2-331312de64c6" (UID: "a49053ac-0a32-49a2-8de2-331312de64c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:41.222783 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222721    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e" (UID: "c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:41.222981 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222963    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49053ac-0a32-49a2-8de2-331312de64c6-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:41.223067 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222985    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46c686c6-ee7b-47f2-933f-a92003727ca6-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:41.223067 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.222999    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:41.224305 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.224284    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a49053ac-0a32-49a2-8de2-331312de64c6-kube-api-access-jn4md" (OuterVolumeSpecName: "kube-api-access-jn4md") pod "a49053ac-0a32-49a2-8de2-331312de64c6" (UID: "a49053ac-0a32-49a2-8de2-331312de64c6"). InnerVolumeSpecName "kube-api-access-jn4md". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:41.224375 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.224324    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46c686c6-ee7b-47f2-933f-a92003727ca6-kube-api-access-7tc52" (OuterVolumeSpecName: "kube-api-access-7tc52") pod "46c686c6-ee7b-47f2-933f-a92003727ca6" (UID: "46c686c6-ee7b-47f2-933f-a92003727ca6"). InnerVolumeSpecName "kube-api-access-7tc52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:41.224741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.224717    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46c686c6-ee7b-47f2-933f-a92003727ca6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46c686c6-ee7b-47f2-933f-a92003727ca6" (UID: "46c686c6-ee7b-47f2-933f-a92003727ca6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:48:41.224741 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.224727    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-kube-api-access-q7n92" (OuterVolumeSpecName: "kube-api-access-q7n92") pod "c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e" (UID: "c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e"). InnerVolumeSpecName "kube-api-access-q7n92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:41.298077 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.297981    2562 generic.go:356] "Generic (PLEG): container finished" podID="c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e" containerID="4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2" exitCode=0
Jul 24 17:48:41.298077 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.298044    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"
Jul 24 17:48:41.298253 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.298067    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc" event={"ID":"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e","Type":"ContainerDied","Data":"4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2"}
Jul 24 17:48:41.298253 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.298123    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc" event={"ID":"c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e","Type":"ContainerDied","Data":"28705adb2e7bbc072a4597bf22958db9ef375a7f3fbb00c343757129dd8a159b"}
Jul 24 17:48:41.298253 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.298149    2562 scope.go:117] "RemoveContainer" containerID="4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2"
Jul 24 17:48:41.299327 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.299306    2562 generic.go:356] "Generic (PLEG): container finished" podID="a49053ac-0a32-49a2-8de2-331312de64c6" containerID="4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477" exitCode=0
Jul 24 17:48:41.299425 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.299330    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h" event={"ID":"a49053ac-0a32-49a2-8de2-331312de64c6","Type":"ContainerDied","Data":"4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477"}
Jul 24 17:48:41.299425 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.299351    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"
Jul 24 17:48:41.299425 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.299373    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h" event={"ID":"a49053ac-0a32-49a2-8de2-331312de64c6","Type":"ContainerDied","Data":"3ba4c3d5045a140f7efb25e6d82a9f90ff10c12ee03cf5cd572e3c61bb343e2a"}
Jul 24 17:48:41.300861 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.300836    2562 generic.go:356] "Generic (PLEG): container finished" podID="46c686c6-ee7b-47f2-933f-a92003727ca6" containerID="7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347" exitCode=0
Jul 24 17:48:41.300988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.300876    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb" event={"ID":"46c686c6-ee7b-47f2-933f-a92003727ca6","Type":"ContainerDied","Data":"7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347"}
Jul 24 17:48:41.300988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.300886    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"
Jul 24 17:48:41.300988 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.300909    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb" event={"ID":"46c686c6-ee7b-47f2-933f-a92003727ca6","Type":"ContainerDied","Data":"f69b32c8653c08c99319c04fe1298b6ff57a420f4b0f040464e2e15bf96c58b3"}
Jul 24 17:48:41.307409 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.307384    2562 scope.go:117] "RemoveContainer" containerID="4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2"
Jul 24 17:48:41.307748 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:48:41.307719    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2\": container with ID starting with 4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2 not found: ID does not exist" containerID="4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2"
Jul 24 17:48:41.307834 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.307747    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2"} err="failed to get container status \"4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2\": rpc error: code = NotFound desc = could not find container \"4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2\": container with ID starting with 4db72f8f2959f6fe683f5cc18a0b9b11af8d86cd10b37af039b80aa053d1b0f2 not found: ID does not exist"
Jul 24 17:48:41.307834 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.307771    2562 scope.go:117] "RemoveContainer" containerID="4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477"
Jul 24 17:48:41.316001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.315986    2562 scope.go:117] "RemoveContainer" containerID="4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477"
Jul 24 17:48:41.316276 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:48:41.316258    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477\": container with ID starting with 4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477 not found: ID does not exist" containerID="4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477"
Jul 24 17:48:41.316351 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.316284    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477"} err="failed to get container status \"4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477\": rpc error: code = NotFound desc = could not find container \"4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477\": container with ID starting with 4b6d853790f6c05adababdd627d0b88b032b543682917f6aeed5f400e3ca5477 not found: ID does not exist"
Jul 24 17:48:41.316351 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.316304    2562 scope.go:117] "RemoveContainer" containerID="7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347"
Jul 24 17:48:41.323001 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.322381    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"]
Jul 24 17:48:41.326403 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.326382    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46c686c6-ee7b-47f2-933f-a92003727ca6-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:41.326492 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.326407    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tc52\" (UniqueName: \"kubernetes.io/projected/46c686c6-ee7b-47f2-933f-a92003727ca6-kube-api-access-7tc52\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:41.326492 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.326420    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn4md\" (UniqueName: \"kubernetes.io/projected/a49053ac-0a32-49a2-8de2-331312de64c6-kube-api-access-jn4md\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:41.326492 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.326433    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7n92\" (UniqueName: \"kubernetes.io/projected/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e-kube-api-access-q7n92\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:41.326677 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.326596    2562 scope.go:117] "RemoveContainer" containerID="7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347"
Jul 24 17:48:41.326904 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:48:41.326875    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347\": container with ID starting with 7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347 not found: ID does not exist" containerID="7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347"
Jul 24 17:48:41.327002 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.326901    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347"} err="failed to get container status \"7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347\": rpc error: code = NotFound desc = could not find container \"7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347\": container with ID starting with 7803f13404314f449dd0284901a6c0fbcf27ade84b0a350074db8dcd36635347 not found: ID does not exist"
Jul 24 17:48:41.327115 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.327011    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-cf30f0-5d75b74cc6-wghtc"]
Jul 24 17:48:41.339200 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.339181    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"]
Jul 24 17:48:41.344192 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.344168    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-cf30f0-ddb7dff9f-wv8rb"]
Jul 24 17:48:41.354835 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.354812    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"]
Jul 24 17:48:41.359090 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.359071    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-cf30f0-545d44c74d-nmn9h"]
Jul 24 17:48:41.505244 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.505221    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46c686c6-ee7b-47f2-933f-a92003727ca6" path="/var/lib/kubelet/pods/46c686c6-ee7b-47f2-933f-a92003727ca6/volumes"
Jul 24 17:48:41.505556 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.505544    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a49053ac-0a32-49a2-8de2-331312de64c6" path="/var/lib/kubelet/pods/a49053ac-0a32-49a2-8de2-331312de64c6/volumes"
Jul 24 17:48:41.505837 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:41.505826    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e" path="/var/lib/kubelet/pods/c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e/volumes"
Jul 24 17:48:52.194993 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:52.194963    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"]
Jul 24 17:48:52.195613 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:52.195245    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy" containerID="cri-o://2c202a3e47d5ef8bce8bea0eaf921cb5787c34d34ee8d886bf0ce68a488dd258" gracePeriod=30
Jul 24 17:48:56.557631 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:56.557597    2562 patch_prober.go:28] interesting pod/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 17:48:56.557999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:56.557654    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 17:48:57.378074 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.378045    2562 generic.go:356] "Generic (PLEG): container finished" podID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerID="2c202a3e47d5ef8bce8bea0eaf921cb5787c34d34ee8d886bf0ce68a488dd258" exitCode=0
Jul 24 17:48:57.378213 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.378121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" event={"ID":"50ebae9b-f44c-4e9f-85f8-c3ddd339c765","Type":"ContainerDied","Data":"2c202a3e47d5ef8bce8bea0eaf921cb5787c34d34ee8d886bf0ce68a488dd258"}
Jul 24 17:48:57.378213 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.378204    2562 scope.go:117] "RemoveContainer" containerID="0296cd522f2e71cd3a974e87e2d396bf568e4c8b5be9daf3408dc0bedad30ee9"
Jul 24 17:48:57.445161 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.445139    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:57.560762 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.560696    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-socket\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.560762 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.560731    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpqzm\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-kube-api-access-fpqzm\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.560762 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.560750    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-data\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.560785    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istiod-ca-cert\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.560910    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-token\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.560970    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-certs\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.560977    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561075    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-podinfo\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561095    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-envoy\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561124    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-credential-socket\") pod \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\" (UID: \"50ebae9b-f44c-4e9f-85f8-c3ddd339c765\") "
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561226    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:57.561356 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561239    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:48:57.561991 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561423    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:57.561991 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561485    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:57.561991 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561506    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:57.561991 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561519    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:57.561991 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.561581    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-data" (OuterVolumeSpecName: "istio-data") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:57.563203 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.563178    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-kube-api-access-fpqzm" (OuterVolumeSpecName: "kube-api-access-fpqzm") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "kube-api-access-fpqzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:57.563331 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.563236    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-token" (OuterVolumeSpecName: "istio-token") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:48:57.563648 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.563624    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:48:57.563749 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.563690    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50ebae9b-f44c-4e9f-85f8-c3ddd339c765" (UID: "50ebae9b-f44c-4e9f-85f8-c3ddd339c765"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:48:57.662204 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.662172    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:57.662204 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.662197    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:57.662204 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.662205    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:57.662204 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.662213    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpqzm\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-kube-api-access-fpqzm\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:57.662464 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.662222    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:57.662464 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:57.662229    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50ebae9b-f44c-4e9f-85f8-c3ddd339c765-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:48:58.390081 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:58.390050    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"
Jul 24 17:48:58.390283 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:58.390049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8" event={"ID":"50ebae9b-f44c-4e9f-85f8-c3ddd339c765","Type":"ContainerDied","Data":"302fae74a962ea87afeacbf8fcc4371b495eaa7160412d1c21034328187ef46e"}
Jul 24 17:48:58.390283 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:58.390203    2562 scope.go:117] "RemoveContainer" containerID="2c202a3e47d5ef8bce8bea0eaf921cb5787c34d34ee8d886bf0ce68a488dd258"
Jul 24 17:48:58.411496 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:58.411470    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"]
Jul 24 17:48:58.415067 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:58.415046    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-cf30f0-openshift-default-665f9d488d-tfnd8"]
Jul 24 17:48:59.504414 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:48:59.504384    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" path="/var/lib/kubelet/pods/50ebae9b-f44c-4e9f-85f8-c3ddd339c765/volumes"
Jul 24 17:49:08.229509 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.229424    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"]
Jul 24 17:49:08.229958 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.229653    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb" podUID="77df28a0-4c9d-4180-ac80-18ef38e4f919" containerName="payload-processing" containerID="cri-o://8fdd554fd938f533765d07fcdf7f74b71c0b25c065d366e9f69c702e97bf2f1e" gracePeriod=30
Jul 24 17:49:08.236589 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.236559    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"]
Jul 24 17:49:08.236861 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.236811    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr" podUID="d08757ad-2b30-4e54-b53c-03f4db7b9507" containerName="payload-pre-processing" containerID="cri-o://74b221057773b6851e45ba888300fd542aa2e9cd9df8402b53869901f9f4f87c" gracePeriod=30
Jul 24 17:49:08.438064 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.437929    2562 generic.go:356] "Generic (PLEG): container finished" podID="77df28a0-4c9d-4180-ac80-18ef38e4f919" containerID="8fdd554fd938f533765d07fcdf7f74b71c0b25c065d366e9f69c702e97bf2f1e" exitCode=0
Jul 24 17:49:08.438064 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.438007    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb" event={"ID":"77df28a0-4c9d-4180-ac80-18ef38e4f919","Type":"ContainerDied","Data":"8fdd554fd938f533765d07fcdf7f74b71c0b25c065d366e9f69c702e97bf2f1e"}
Jul 24 17:49:08.439643 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.439622    2562 generic.go:356] "Generic (PLEG): container finished" podID="d08757ad-2b30-4e54-b53c-03f4db7b9507" containerID="74b221057773b6851e45ba888300fd542aa2e9cd9df8402b53869901f9f4f87c" exitCode=0
Jul 24 17:49:08.439754 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.439674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr" event={"ID":"d08757ad-2b30-4e54-b53c-03f4db7b9507","Type":"ContainerDied","Data":"74b221057773b6851e45ba888300fd542aa2e9cd9df8402b53869901f9f4f87c"}
Jul 24 17:49:08.495386 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.495326    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:49:08.525544 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.525519    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:49:08.545803 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.545775    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77df28a0-4c9d-4180-ac80-18ef38e4f919-plugins-config-volume\") pod \"77df28a0-4c9d-4180-ac80-18ef38e4f919\" (UID: \"77df28a0-4c9d-4180-ac80-18ef38e4f919\") "
Jul 24 17:49:08.545935 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.545848    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwkrl\" (UniqueName: \"kubernetes.io/projected/77df28a0-4c9d-4180-ac80-18ef38e4f919-kube-api-access-hwkrl\") pod \"77df28a0-4c9d-4180-ac80-18ef38e4f919\" (UID: \"77df28a0-4c9d-4180-ac80-18ef38e4f919\") "
Jul 24 17:49:08.546146 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.546125    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77df28a0-4c9d-4180-ac80-18ef38e4f919-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77df28a0-4c9d-4180-ac80-18ef38e4f919" (UID: "77df28a0-4c9d-4180-ac80-18ef38e4f919"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:49:08.546219 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.546191    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77df28a0-4c9d-4180-ac80-18ef38e4f919-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:08.548315 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.548292    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77df28a0-4c9d-4180-ac80-18ef38e4f919-kube-api-access-hwkrl" (OuterVolumeSpecName: "kube-api-access-hwkrl") pod "77df28a0-4c9d-4180-ac80-18ef38e4f919" (UID: "77df28a0-4c9d-4180-ac80-18ef38e4f919"). InnerVolumeSpecName "kube-api-access-hwkrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:08.647325 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.647301    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08757ad-2b30-4e54-b53c-03f4db7b9507-plugins-config-volume\") pod \"d08757ad-2b30-4e54-b53c-03f4db7b9507\" (UID: \"d08757ad-2b30-4e54-b53c-03f4db7b9507\") "
Jul 24 17:49:08.647436 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.647384    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff7w8\" (UniqueName: \"kubernetes.io/projected/d08757ad-2b30-4e54-b53c-03f4db7b9507-kube-api-access-ff7w8\") pod \"d08757ad-2b30-4e54-b53c-03f4db7b9507\" (UID: \"d08757ad-2b30-4e54-b53c-03f4db7b9507\") "
Jul 24 17:49:08.647512 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.647500    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwkrl\" (UniqueName: \"kubernetes.io/projected/77df28a0-4c9d-4180-ac80-18ef38e4f919-kube-api-access-hwkrl\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:08.647705 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.647681    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d08757ad-2b30-4e54-b53c-03f4db7b9507-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d08757ad-2b30-4e54-b53c-03f4db7b9507" (UID: "d08757ad-2b30-4e54-b53c-03f4db7b9507"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:49:08.649409 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.649390    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d08757ad-2b30-4e54-b53c-03f4db7b9507-kube-api-access-ff7w8" (OuterVolumeSpecName: "kube-api-access-ff7w8") pod "d08757ad-2b30-4e54-b53c-03f4db7b9507" (UID: "d08757ad-2b30-4e54-b53c-03f4db7b9507"). InnerVolumeSpecName "kube-api-access-ff7w8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:08.747847 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.747792    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff7w8\" (UniqueName: \"kubernetes.io/projected/d08757ad-2b30-4e54-b53c-03f4db7b9507-kube-api-access-ff7w8\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:08.747847 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:08.747809    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d08757ad-2b30-4e54-b53c-03f4db7b9507-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:09.445581 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.445552    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"
Jul 24 17:49:09.446064 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.445596    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr" event={"ID":"d08757ad-2b30-4e54-b53c-03f4db7b9507","Type":"ContainerDied","Data":"5a4407596e7f3f2179fca1a42e9cea16553e6ed4f4f995ed1b5c4828fd8333aa"}
Jul 24 17:49:09.446064 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.445665    2562 scope.go:117] "RemoveContainer" containerID="74b221057773b6851e45ba888300fd542aa2e9cd9df8402b53869901f9f4f87c"
Jul 24 17:49:09.446991 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.446972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb" event={"ID":"77df28a0-4c9d-4180-ac80-18ef38e4f919","Type":"ContainerDied","Data":"c55db154564f3480bd21faa34aa707739d9aeb9e88575547c2a12d6b2383c03e"}
Jul 24 17:49:09.447092 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.447037    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"
Jul 24 17:49:09.455572 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.455557    2562 scope.go:117] "RemoveContainer" containerID="8fdd554fd938f533765d07fcdf7f74b71c0b25c065d366e9f69c702e97bf2f1e"
Jul 24 17:49:09.467849 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.467829    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"]
Jul 24 17:49:09.471646 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.471624    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0e9e3f-889476-pvtmr"]
Jul 24 17:49:09.483115 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.483087    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"]
Jul 24 17:49:09.486882 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.486861    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0e9e3f-565485bfc4-nvjnb"]
Jul 24 17:49:09.507278 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.507251    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77df28a0-4c9d-4180-ac80-18ef38e4f919" path="/var/lib/kubelet/pods/77df28a0-4c9d-4180-ac80-18ef38e4f919/volumes"
Jul 24 17:49:09.507609 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:09.507597    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d08757ad-2b30-4e54-b53c-03f4db7b9507" path="/var/lib/kubelet/pods/d08757ad-2b30-4e54-b53c-03f4db7b9507/volumes"
Jul 24 17:49:19.192162 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:19.192128    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"]
Jul 24 17:49:19.192612 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:19.192361    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy" containerID="cri-o://e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c" gracePeriod=30
Jul 24 17:49:19.589164 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:19.589132    2562 patch_prober.go:28] interesting pod/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 17:49:19.589342 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:19.589206    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 17:49:20.067439 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067399    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"]
Jul 24 17:49:20.067734 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067721    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46c686c6-ee7b-47f2-933f-a92003727ca6" containerName="maas-api"
Jul 24 17:49:20.067734 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067734    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="46c686c6-ee7b-47f2-933f-a92003727ca6" containerName="maas-api"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067742    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77df28a0-4c9d-4180-ac80-18ef38e4f919" containerName="payload-processing"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067748    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77df28a0-4c9d-4180-ac80-18ef38e4f919" containerName="payload-processing"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067757    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067763    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067779    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a49053ac-0a32-49a2-8de2-331312de64c6" containerName="payload-processing"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067785    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a49053ac-0a32-49a2-8de2-331312de64c6" containerName="payload-processing"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067794    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e" containerName="payload-pre-processing"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067799    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e" containerName="payload-pre-processing"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067805    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d08757ad-2b30-4e54-b53c-03f4db7b9507" containerName="payload-pre-processing"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067810    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d08757ad-2b30-4e54-b53c-03f4db7b9507" containerName="payload-pre-processing"
Jul 24 17:49:20.067856 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067857    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy"
Jul 24 17:49:20.068432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067864    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy"
Jul 24 17:49:20.068432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067871    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7287be2-5f49-4ba8-8a0b-91f5d5e36f2e" containerName="payload-pre-processing"
Jul 24 17:49:20.068432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067878    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a49053ac-0a32-49a2-8de2-331312de64c6" containerName="payload-processing"
Jul 24 17:49:20.068432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067883    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="46c686c6-ee7b-47f2-933f-a92003727ca6" containerName="maas-api"
Jul 24 17:49:20.068432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067889    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77df28a0-4c9d-4180-ac80-18ef38e4f919" containerName="payload-processing"
Jul 24 17:49:20.068432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.067894    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d08757ad-2b30-4e54-b53c-03f4db7b9507" containerName="payload-pre-processing"
Jul 24 17:49:20.070948 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.070930    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.074194 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.074172    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-bb94b3-openshift-default-dockercfg-9t6h7\""
Jul 24 17:49:20.085668 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.085645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"]
Jul 24 17:49:20.150211 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150167    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-certs\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.150211 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150203    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-data\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.150470 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150294    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-envoy\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.150470 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150330    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c95dd0-732f-4362-a869-d2484c966208-istio-podinfo\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.150470 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150389    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-istio-token\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.150470 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150412    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c95dd0-732f-4362-a869-d2484c966208-istiod-ca-cert\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.150470 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nxns\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-kube-api-access-4nxns\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.150735 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150509    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-socket\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.150735 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.150541    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-credential-socket\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.251606 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-certs\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.251606 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251604    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-data\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-envoy\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251653    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c95dd0-732f-4362-a869-d2484c966208-istio-podinfo\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251696    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-istio-token\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c95dd0-732f-4362-a869-d2484c966208-istiod-ca-cert\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251754    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nxns\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-kube-api-access-4nxns\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251788    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-socket\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.251818    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-credential-socket\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.252045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-certs\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252176 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.252092    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-data\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.252391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-socket\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252596 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.252423    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-credential-socket\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.252659 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.252616    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c95dd0-732f-4362-a869-d2484c966208-istiod-ca-cert\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.254601 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.254581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-envoy\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.254812 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.254793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c95dd0-732f-4362-a869-d2484c966208-istio-podinfo\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.265290 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.265271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-istio-token\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.270391 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.270369    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nxns\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-kube-api-access-4nxns\") pod \"e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") " pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.381169 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.381068    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:20.522285 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.522259    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"]
Jul 24 17:49:20.526087 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:49:20.526057    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c95dd0_732f_4362_a869_d2484c966208.slice/crio-5e69eac713ae7b10ff7299b676e40184e6ae3b19db715dbf9674ae12daa2fe50 WatchSource:0}: Error finding container 5e69eac713ae7b10ff7299b676e40184e6ae3b19db715dbf9674ae12daa2fe50: Status 404 returned error can't find the container with id 5e69eac713ae7b10ff7299b676e40184e6ae3b19db715dbf9674ae12daa2fe50
Jul 24 17:49:20.528146 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.528114    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:20.528230 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.528176    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:20.528230 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:20.528206    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:21.503810 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:21.503774    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" event={"ID":"98c95dd0-732f-4362-a869-d2484c966208","Type":"ContainerStarted","Data":"743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6"}
Jul 24 17:49:21.504193 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:21.503827    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" event={"ID":"98c95dd0-732f-4362-a869-d2484c966208","Type":"ContainerStarted","Data":"5e69eac713ae7b10ff7299b676e40184e6ae3b19db715dbf9674ae12daa2fe50"}
Jul 24 17:49:21.526928 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:21.526877    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" podStartSLOduration=1.5268606679999999 podStartE2EDuration="1.526860668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:21.525832525 +0000 UTC m=+2350.496964717" watchObservedRunningTime="2026-07-24 17:49:21.526860668 +0000 UTC m=+2350.497992848"
Jul 24 17:49:22.381968 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:22.381933    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:22.386867 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:22.386840    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:22.503916 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:22.503885    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:22.504869 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:22.504852    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:23.619209 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.619174    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"]
Jul 24 17:49:23.619699 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.619482    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy"
Jul 24 17:49:23.619699 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.619492    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="50ebae9b-f44c-4e9f-85f8-c3ddd339c765" containerName="istio-proxy"
Jul 24 17:49:23.622288 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.622268    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.625193 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.625153    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 17:49:23.625193 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.625171    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7gnds\""
Jul 24 17:49:23.626364 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.626343    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 17:49:23.626503 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.626418    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-bb94b3-serving-cert\""
Jul 24 17:49:23.636152 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.636127    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"]
Jul 24 17:49:23.781817 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.781781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjkbj\" (UniqueName: \"kubernetes.io/projected/1dfa0922-d592-48aa-adff-e3a542c54e5b-kube-api-access-cjkbj\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.781817 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.781818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfa0922-d592-48aa-adff-e3a542c54e5b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.782173 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.781853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfa0922-d592-48aa-adff-e3a542c54e5b-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.883301 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.883208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjkbj\" (UniqueName: \"kubernetes.io/projected/1dfa0922-d592-48aa-adff-e3a542c54e5b-kube-api-access-cjkbj\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.883301 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.883250    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfa0922-d592-48aa-adff-e3a542c54e5b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.883301 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.883293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfa0922-d592-48aa-adff-e3a542c54e5b-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.883911 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.883889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfa0922-d592-48aa-adff-e3a542c54e5b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.886002 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.885970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfa0922-d592-48aa-adff-e3a542c54e5b-maas-api-tls\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.893589 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.893565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjkbj\" (UniqueName: \"kubernetes.io/projected/1dfa0922-d592-48aa-adff-e3a542c54e5b-kube-api-access-cjkbj\") pod \"maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:23.935183 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:23.935149    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:24.082237 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.082204    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"]
Jul 24 17:49:24.085434 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:49:24.085398    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dfa0922_d592_48aa_adff_e3a542c54e5b.slice/crio-d313bba8736042cbfd2adc281e536bc593c87d691ad22e5fa4b8b70a18835242 WatchSource:0}: Error finding container d313bba8736042cbfd2adc281e536bc593c87d691ad22e5fa4b8b70a18835242: Status 404 returned error can't find the container with id d313bba8736042cbfd2adc281e536bc593c87d691ad22e5fa4b8b70a18835242
Jul 24 17:49:24.439859 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.439836    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:49:24.501763 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.501670    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"]
Jul 24 17:49:24.501977 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.501962    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy"
Jul 24 17:49:24.502094 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.501978    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy"
Jul 24 17:49:24.502164 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.502096    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy"
Jul 24 17:49:24.502164 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.502107    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy"
Jul 24 17:49:24.504761 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.504746    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:24.507475 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.507452    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-bb94b3\""
Jul 24 17:49:24.514207 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.514181    2562 generic.go:356] "Generic (PLEG): container finished" podID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerID="e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c" exitCode=0
Jul 24 17:49:24.514306 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.514223    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"
Jul 24 17:49:24.514306 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.514264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" event={"ID":"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d","Type":"ContainerDied","Data":"e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c"}
Jul 24 17:49:24.514414 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.514320    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6" event={"ID":"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d","Type":"ContainerDied","Data":"5d05f7aa615aea306f0734a3f1ffd98223394ebf17c4356be7b186bff34d72ac"}
Jul 24 17:49:24.514414 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.514350    2562 scope.go:117] "RemoveContainer" containerID="e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c"
Jul 24 17:49:24.515690 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.515664    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv" event={"ID":"1dfa0922-d592-48aa-adff-e3a542c54e5b","Type":"ContainerStarted","Data":"d313bba8736042cbfd2adc281e536bc593c87d691ad22e5fa4b8b70a18835242"}
Jul 24 17:49:24.523630 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.523610    2562 scope.go:117] "RemoveContainer" containerID="9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8"
Jul 24 17:49:24.524999 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.524981    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"]
Jul 24 17:49:24.532827 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.532806    2562 scope.go:117] "RemoveContainer" containerID="e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c"
Jul 24 17:49:24.533191 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:49:24.533169    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c\": container with ID starting with e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c not found: ID does not exist" containerID="e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c"
Jul 24 17:49:24.533271 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.533203    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c"} err="failed to get container status \"e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c\": rpc error: code = NotFound desc = could not find container \"e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c\": container with ID starting with e0116171bfee0a47042f0203e1ad689118bba5304319c13d0686ad184fba1b4c not found: ID does not exist"
Jul 24 17:49:24.533271 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.533226    2562 scope.go:117] "RemoveContainer" containerID="9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8"
Jul 24 17:49:24.533588 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:49:24.533561    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8\": container with ID starting with 9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8 not found: ID does not exist" containerID="9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8"
Jul 24 17:49:24.533664 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.533595    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8"} err="failed to get container status \"9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8\": rpc error: code = NotFound desc = could not find container \"9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8\": container with ID starting with 9b4d1f94b01c228e5eb180fb9550d200e5d6307a1afdc433bdaec22e6dbd92e8 not found: ID does not exist"
Jul 24 17:49:24.575316 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.575288    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"]
Jul 24 17:49:24.575621 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.575609    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy"
Jul 24 17:49:24.575665 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.575623    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" containerName="istio-proxy"
Jul 24 17:49:24.578714 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.578697    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:24.582111 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.582093    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-bb94b3-dockercfg-nxsck\""
Jul 24 17:49:24.590095 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590075    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-socket\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590223 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590117    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istiod-ca-cert\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590223 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590156    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-certs\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590223 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590198    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-envoy\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590223    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-credential-socket\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590253    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w599\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-kube-api-access-9w599\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590324    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-data\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590362    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-token\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590354    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:49:24.590399 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590393    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-podinfo\") pod \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\" (UID: \"e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d\") "
Jul 24 17:49:24.590698 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590459    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:49:24.590698 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590529    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:49:24.590698 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590562    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2519ca4-8872-4038-bb16-f2da6f087073-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk\" (UID: \"a2519ca4-8872-4038-bb16-f2da6f087073\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:24.590852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpkzp\" (UniqueName: \"kubernetes.io/projected/a2519ca4-8872-4038-bb16-f2da6f087073-kube-api-access-gpkzp\") pod \"payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk\" (UID: \"a2519ca4-8872-4038-bb16-f2da6f087073\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:24.590852 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590833    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:49:24.590955 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590863    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.590955 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590880    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.590955 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590893    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.590955 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.590877    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"]
Jul 24 17:49:24.591365 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.591339    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:49:24.593286 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.593262    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:24.593426 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.593407    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-kube-api-access-9w599" (OuterVolumeSpecName: "kube-api-access-9w599") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "kube-api-access-9w599". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:49:24.593610 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.593586    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:49:24.593706 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.593589    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" (UID: "e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 17:49:24.692041 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.691985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d971647a-5331-4212-b720-2f5f90243930-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j\" (UID: \"d971647a-5331-4212-b720-2f5f90243930\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692133    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpkzp\" (UniqueName: \"kubernetes.io/projected/a2519ca4-8872-4038-bb16-f2da6f087073-kube-api-access-gpkzp\") pod \"payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk\" (UID: \"a2519ca4-8872-4038-bb16-f2da6f087073\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2519ca4-8872-4038-bb16-f2da6f087073-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk\" (UID: \"a2519ca4-8872-4038-bb16-f2da6f087073\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692218    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2fbn\" (UniqueName: \"kubernetes.io/projected/d971647a-5331-4212-b720-2f5f90243930-kube-api-access-x2fbn\") pod \"payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j\" (UID: \"d971647a-5331-4212-b720-2f5f90243930\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692255    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692277    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692287    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w599\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-kube-api-access-9w599\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692296    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692304    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.692430 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692312    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:49:24.692940 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.692920    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2519ca4-8872-4038-bb16-f2da6f087073-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk\" (UID: \"a2519ca4-8872-4038-bb16-f2da6f087073\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:24.704632 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.704599    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpkzp\" (UniqueName: \"kubernetes.io/projected/a2519ca4-8872-4038-bb16-f2da6f087073-kube-api-access-gpkzp\") pod \"payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk\" (UID: \"a2519ca4-8872-4038-bb16-f2da6f087073\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:24.793134 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.793041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d971647a-5331-4212-b720-2f5f90243930-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j\" (UID: \"d971647a-5331-4212-b720-2f5f90243930\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:24.793297 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.793185    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2fbn\" (UniqueName: \"kubernetes.io/projected/d971647a-5331-4212-b720-2f5f90243930-kube-api-access-x2fbn\") pod \"payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j\" (UID: \"d971647a-5331-4212-b720-2f5f90243930\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:24.793776 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.793758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d971647a-5331-4212-b720-2f5f90243930-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j\" (UID: \"d971647a-5331-4212-b720-2f5f90243930\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:24.807359 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.807327    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2fbn\" (UniqueName: \"kubernetes.io/projected/d971647a-5331-4212-b720-2f5f90243930-kube-api-access-x2fbn\") pod \"payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j\" (UID: \"d971647a-5331-4212-b720-2f5f90243930\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:24.815186 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.815156    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:24.841151 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.841109    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"]
Jul 24 17:49:24.847138 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.847107    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0e9e3f-openshift-default-d795749b6-zlzd6"]
Jul 24 17:49:24.887964 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.887933    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:24.980979 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:24.980942    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"]
Jul 24 17:49:25.062838 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.062812    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"]
Jul 24 17:49:25.065785 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:49:25.065758    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd971647a_5331_4212_b720_2f5f90243930.slice/crio-20d63963457984545ed31d24f202d5a6ad47785fd95230a685030f20f5414d02 WatchSource:0}: Error finding container 20d63963457984545ed31d24f202d5a6ad47785fd95230a685030f20f5414d02: Status 404 returned error can't find the container with id 20d63963457984545ed31d24f202d5a6ad47785fd95230a685030f20f5414d02
Jul 24 17:49:25.505911 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.505877    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d" path="/var/lib/kubelet/pods/e20bd451-2b2b-41b2-8fe3-f2b58ee8e80d/volumes"
Jul 24 17:49:25.520866 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.520834    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j" event={"ID":"d971647a-5331-4212-b720-2f5f90243930","Type":"ContainerStarted","Data":"cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559"}
Jul 24 17:49:25.521076 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.520962    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:25.521076 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.521004    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j" event={"ID":"d971647a-5331-4212-b720-2f5f90243930","Type":"ContainerStarted","Data":"20d63963457984545ed31d24f202d5a6ad47785fd95230a685030f20f5414d02"}
Jul 24 17:49:25.523295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.523269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv" event={"ID":"1dfa0922-d592-48aa-adff-e3a542c54e5b","Type":"ContainerStarted","Data":"27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e"}
Jul 24 17:49:25.523418 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.523355    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:25.524949 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.524928    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk" event={"ID":"a2519ca4-8872-4038-bb16-f2da6f087073","Type":"ContainerStarted","Data":"1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9"}
Jul 24 17:49:25.525088 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.524961    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk" event={"ID":"a2519ca4-8872-4038-bb16-f2da6f087073","Type":"ContainerStarted","Data":"5fe3df115750b63e2a10e4ba2722620311e4fe9125b23c0ed8f9060001bf2509"}
Jul 24 17:49:25.525088 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.525051    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:25.543506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.543465    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j" podStartSLOduration=1.543452577 podStartE2EDuration="1.543452577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:25.542080385 +0000 UTC m=+2354.513212557" watchObservedRunningTime="2026-07-24 17:49:25.543452577 +0000 UTC m=+2354.514584748"
Jul 24 17:49:25.575884 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.575830    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv" podStartSLOduration=2.158665116 podStartE2EDuration="2.575813376s" totalImagesPullingTime="417.14826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:25.574708257 +0000 UTC m=+2354.545840433" watchObservedRunningTime="2026-07-24 17:49:25.575813376 +0000 UTC m=+2354.546945556"
Jul 24 17:49:25.607831 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:25.607781    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk" podStartSLOduration=1.607765949 podStartE2EDuration="1.607765949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:25.606655774 +0000 UTC m=+2354.577787954" watchObservedRunningTime="2026-07-24 17:49:25.607765949 +0000 UTC m=+2354.578898130"
Jul 24 17:49:28.630491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.630463    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"]
Jul 24 17:49:28.634910 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.634886    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.649563 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.649536    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-5c7cfc-openshift-default-dockercfg-jjjts\""
Jul 24 17:49:28.662776 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.662750    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"]
Jul 24 17:49:28.727385 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.727347    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-envoy\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.727584 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.727460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ee3260-69e7-4c59-b4f4-60ea81eed431-istiod-ca-cert\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.727584 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.727497    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-data\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.727754 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.727684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-credential-socket\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.727754 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.727715    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-token\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.727872 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.727791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qpzl\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-kube-api-access-8qpzl\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.727872 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.727864    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-podinfo\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.728038 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.727949    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-certs\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.728105 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.728060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-socket\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829169 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-credential-socket\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829169 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-token\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829443 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829191    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qpzl\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-kube-api-access-8qpzl\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829443 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-podinfo\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829443 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-certs\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829443 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829386    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-socket\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829443 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-envoy\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829712 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829498    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ee3260-69e7-4c59-b4f4-60ea81eed431-istiod-ca-cert\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829712 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-data\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829712 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-credential-socket\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829871 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-certs\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829871 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-socket\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.829978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.829904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-data\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.830250 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.830230    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ee3260-69e7-4c59-b4f4-60ea81eed431-istiod-ca-cert\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.831751 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.831721    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-envoy\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.832013 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.831995    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-podinfo\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.838054 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.838034    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-token\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.838377 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.838358    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qpzl\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-kube-api-access-8qpzl\") pod \"e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") " pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:28.946738 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:28.946712    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:29.089137 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:29.089111    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"]
Jul 24 17:49:29.091981 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:49:29.091954    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ee3260_69e7_4c59_b4f4_60ea81eed431.slice/crio-46a4e00821b7010c4a850f45c8fb070636242e5d3de285a865a4fcf304d6d4cb WatchSource:0}: Error finding container 46a4e00821b7010c4a850f45c8fb070636242e5d3de285a865a4fcf304d6d4cb: Status 404 returned error can't find the container with id 46a4e00821b7010c4a850f45c8fb070636242e5d3de285a865a4fcf304d6d4cb
Jul 24 17:49:29.094679 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:29.094643    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:29.094768 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:29.094718    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:29.094768 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:29.094747    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:29.543431 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:29.543395    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" event={"ID":"12ee3260-69e7-4c59-b4f4-60ea81eed431","Type":"ContainerStarted","Data":"3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e"}
Jul 24 17:49:29.543656 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:29.543450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" event={"ID":"12ee3260-69e7-4c59-b4f4-60ea81eed431","Type":"ContainerStarted","Data":"46a4e00821b7010c4a850f45c8fb070636242e5d3de285a865a4fcf304d6d4cb"}
Jul 24 17:49:29.570657 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:29.570605    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" podStartSLOduration=1.570589375 podStartE2EDuration="1.570589375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:29.56764227 +0000 UTC m=+2358.538774451" watchObservedRunningTime="2026-07-24 17:49:29.570589375 +0000 UTC m=+2358.541721556"
Jul 24 17:49:29.947535 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:29.947511    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:30.952196 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:30.952161    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:31.534839 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:31.534810    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 17:49:31.552371 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:31.552343    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:31.553774 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:31.553753    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:32.371501 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.371461    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"]
Jul 24 17:49:32.376098 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.376072    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:32.379763 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.379510    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-5c7cfc\""
Jul 24 17:49:32.387760 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.387733    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"]
Jul 24 17:49:32.467335 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.467279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ng82\" (UniqueName: \"kubernetes.io/projected/e3253f25-a9d7-4364-9b92-21e17d22e9f3-kube-api-access-6ng82\") pod \"payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl\" (UID: \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:32.467546 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.467351    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3253f25-a9d7-4364-9b92-21e17d22e9f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl\" (UID: \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:32.531214 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.531178    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"]
Jul 24 17:49:32.534401 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.534387    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:32.539094 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.539062    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-5c7cfc-dockercfg-4vddc\""
Jul 24 17:49:32.553597 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.553554    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"]
Jul 24 17:49:32.568949 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.568914    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ng82\" (UniqueName: \"kubernetes.io/projected/e3253f25-a9d7-4364-9b92-21e17d22e9f3-kube-api-access-6ng82\") pod \"payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl\" (UID: \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:32.569244 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.569220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3253f25-a9d7-4364-9b92-21e17d22e9f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl\" (UID: \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:32.570333 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.570308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3253f25-a9d7-4364-9b92-21e17d22e9f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl\" (UID: \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:32.587871 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.587843    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ng82\" (UniqueName: \"kubernetes.io/projected/e3253f25-a9d7-4364-9b92-21e17d22e9f3-kube-api-access-6ng82\") pod \"payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl\" (UID: \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:32.671009 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.670907    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503b5993-5781-46b0-8ea5-c328eb0905be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8\" (UID: \"503b5993-5781-46b0-8ea5-c328eb0905be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:32.671009 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.670958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzjh4\" (UniqueName: \"kubernetes.io/projected/503b5993-5781-46b0-8ea5-c328eb0905be-kube-api-access-qzjh4\") pod \"payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8\" (UID: \"503b5993-5781-46b0-8ea5-c328eb0905be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:32.689862 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.689831    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:32.781042 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.778715    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503b5993-5781-46b0-8ea5-c328eb0905be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8\" (UID: \"503b5993-5781-46b0-8ea5-c328eb0905be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:32.781042 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.778765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzjh4\" (UniqueName: \"kubernetes.io/projected/503b5993-5781-46b0-8ea5-c328eb0905be-kube-api-access-qzjh4\") pod \"payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8\" (UID: \"503b5993-5781-46b0-8ea5-c328eb0905be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:32.781042 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.779979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503b5993-5781-46b0-8ea5-c328eb0905be-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8\" (UID: \"503b5993-5781-46b0-8ea5-c328eb0905be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:32.790175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.790145    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzjh4\" (UniqueName: \"kubernetes.io/projected/503b5993-5781-46b0-8ea5-c328eb0905be-kube-api-access-qzjh4\") pod \"payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8\" (UID: \"503b5993-5781-46b0-8ea5-c328eb0905be\") " pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:32.844404 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.844374    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:32.868046 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:32.866031    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"]
Jul 24 17:49:33.040723 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.040701    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"]
Jul 24 17:49:33.042752 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:49:33.042724    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503b5993_5781_46b0_8ea5_c328eb0905be.slice/crio-4b6c26cffca710e3fd5e4b562c71fc01739bb4dc4e315e10089f6dee130f5a8b WatchSource:0}: Error finding container 4b6c26cffca710e3fd5e4b562c71fc01739bb4dc4e315e10089f6dee130f5a8b: Status 404 returned error can't find the container with id 4b6c26cffca710e3fd5e4b562c71fc01739bb4dc4e315e10089f6dee130f5a8b
Jul 24 17:49:33.571175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.570049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl" event={"ID":"e3253f25-a9d7-4364-9b92-21e17d22e9f3","Type":"ContainerStarted","Data":"3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec"}
Jul 24 17:49:33.571175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.570115    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl" event={"ID":"e3253f25-a9d7-4364-9b92-21e17d22e9f3","Type":"ContainerStarted","Data":"bec34a8e1d68e974dad41b0b34689db2e8f616ec0090fceb523646e523921070"}
Jul 24 17:49:33.571175 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.570881    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:33.574300 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.574156    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8" event={"ID":"503b5993-5781-46b0-8ea5-c328eb0905be","Type":"ContainerStarted","Data":"77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f"}
Jul 24 17:49:33.574300 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.574199    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8" event={"ID":"503b5993-5781-46b0-8ea5-c328eb0905be","Type":"ContainerStarted","Data":"4b6c26cffca710e3fd5e4b562c71fc01739bb4dc4e315e10089f6dee130f5a8b"}
Jul 24 17:49:33.574300 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.574228    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:33.595820 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.595774    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl" podStartSLOduration=1.5957583990000002 podStartE2EDuration="1.595758399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:33.595298798 +0000 UTC m=+2362.566430982" watchObservedRunningTime="2026-07-24 17:49:33.595758399 +0000 UTC m=+2362.566890579"
Jul 24 17:49:33.618495 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:33.618451    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8" podStartSLOduration=1.618436307 podStartE2EDuration="1.618436307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:33.616996422 +0000 UTC m=+2362.588128602" watchObservedRunningTime="2026-07-24 17:49:33.618436307 +0000 UTC m=+2362.589568489"
Jul 24 17:49:36.531142 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:36.531088    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j" podUID="d971647a-5331-4212-b720-2f5f90243930" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.66:9004: connect: connection refused"
Jul 24 17:49:36.531744 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:36.531722    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 17:49:38.546267 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:38.546220    2562 patch_prober.go:28] interesting pod/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 17:49:38.546863 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:38.546826    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 17:49:39.606185 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:39.606095    2562 generic.go:356] "Generic (PLEG): container finished" podID="98c95dd0-732f-4362-a869-d2484c966208" containerID="743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6" exitCode=0
Jul 24 17:49:39.606185 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:39.606130    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" event={"ID":"98c95dd0-732f-4362-a869-d2484c966208","Type":"ContainerDied","Data":"743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6"}
Jul 24 17:49:39.606630 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:39.606542    2562 scope.go:117] "RemoveContainer" containerID="743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6"
Jul 24 17:49:39.611069 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:39.608285    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:39.611069 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:39.608524    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:39.611069 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:39.608665    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:40.616366 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:40.616327    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" event={"ID":"98c95dd0-732f-4362-a869-d2484c966208","Type":"ContainerStarted","Data":"e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c"}
Jul 24 17:49:41.382493 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:41.382456    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:41.382929 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:41.382842    2562 patch_prober.go:28] interesting pod/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 24 17:49:41.383147 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:41.383119    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 24 17:49:42.382553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:42.382505    2562 patch_prober.go:28] interesting pod/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 24 17:49:42.383086 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:42.382579    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 24 17:49:43.381962 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:43.381921    2562 patch_prober.go:28] interesting pod/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 24 17:49:43.382182 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:43.381986    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 24 17:49:44.382178 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:44.382141    2562 patch_prober.go:28] interesting pod/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 24 17:49:44.382564 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:44.382200    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 24 17:49:44.583061 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:44.583005    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 17:49:45.385375 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:45.385317    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:45.385925 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:45.385624    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:45.386679 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:45.386653    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 17:49:45.588643 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:45.588618    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 17:49:46.530132 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.530104    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 17:49:46.553786 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.553748    2562 patch_prober.go:28] interesting pod/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 24 17:49:46.553944 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.553807    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 24 17:49:46.648700 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.648662    2562 generic.go:356] "Generic (PLEG): container finished" podID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerID="3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e" exitCode=0
Jul 24 17:49:46.648892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.648728    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" event={"ID":"12ee3260-69e7-4c59-b4f4-60ea81eed431","Type":"ContainerDied","Data":"3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e"}
Jul 24 17:49:46.649239 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.649221    2562 scope.go:117] "RemoveContainer" containerID="3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e"
Jul 24 17:49:46.650215 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.650186    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:46.650306 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.650254    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:46.650306 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:46.650294    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 24 17:49:47.657396 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:47.657352    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" event={"ID":"12ee3260-69e7-4c59-b4f4-60ea81eed431","Type":"ContainerStarted","Data":"d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721"}
Jul 24 17:49:47.947703 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:47.947674    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:47.947929 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:47.947909    2562 patch_prober.go:28] interesting pod/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 24 17:49:47.947986 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:47.947954    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 24 17:49:48.947676 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:48.947633    2562 patch_prober.go:28] interesting pod/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 24 17:49:48.948138 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:48.947691    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 24 17:49:49.947803 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:49.947760    2562 patch_prober.go:28] interesting pod/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 24 17:49:49.948249 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:49.947824    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 24 17:49:50.947330 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:50.947289    2562 patch_prober.go:28] interesting pod/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 24 17:49:50.947531 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:50.947350    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 24 17:49:51.951227 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:51.951196    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:51.951631 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:51.951521    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:49:51.952359 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:49:51.952338    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 17:50:11.579292 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:50:11.579264    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:50:11.583946 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:50:11.583922    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:55:11.604040 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:55:11.603944    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:55:11.610281 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:55:11.610259    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 17:56:30.908740 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:30.908700    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"]
Jul 24 17:56:30.912287 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:30.912270    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:30.915721 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:30.915696    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-8a6414-dockercfg-fbjp9\""
Jul 24 17:56:30.915861 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:30.915736    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-8a6414\""
Jul 24 17:56:30.932432 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:30.932406    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"]
Jul 24 17:56:30.997461 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:30.997425    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2zq8\" (UniqueName: \"kubernetes.io/projected/186c8710-409a-44ac-a5ab-fd57bb540034-kube-api-access-w2zq8\") pod \"payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k\" (UID: \"186c8710-409a-44ac-a5ab-fd57bb540034\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:30.997675 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:30.997647    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/186c8710-409a-44ac-a5ab-fd57bb540034-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k\" (UID: \"186c8710-409a-44ac-a5ab-fd57bb540034\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:31.098941 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:31.098908    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2zq8\" (UniqueName: \"kubernetes.io/projected/186c8710-409a-44ac-a5ab-fd57bb540034-kube-api-access-w2zq8\") pod \"payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k\" (UID: \"186c8710-409a-44ac-a5ab-fd57bb540034\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:31.099150 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:31.099002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/186c8710-409a-44ac-a5ab-fd57bb540034-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k\" (UID: \"186c8710-409a-44ac-a5ab-fd57bb540034\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:31.099654 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:31.099635    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/186c8710-409a-44ac-a5ab-fd57bb540034-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k\" (UID: \"186c8710-409a-44ac-a5ab-fd57bb540034\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:31.109149 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:31.109110    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2zq8\" (UniqueName: \"kubernetes.io/projected/186c8710-409a-44ac-a5ab-fd57bb540034-kube-api-access-w2zq8\") pod \"payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k\" (UID: \"186c8710-409a-44ac-a5ab-fd57bb540034\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:31.222627 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:31.222594    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:31.562070 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:31.562045    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"]
Jul 24 17:56:31.565177 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:56:31.565136    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186c8710_409a_44ac_a5ab_fd57bb540034.slice/crio-b24e9719344b00d762535092b2989abdaa34497189be91076b005823fcd47e80 WatchSource:0}: Error finding container b24e9719344b00d762535092b2989abdaa34497189be91076b005823fcd47e80: Status 404 returned error can't find the container with id b24e9719344b00d762535092b2989abdaa34497189be91076b005823fcd47e80
Jul 24 17:56:32.477542 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:32.477509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k" event={"ID":"186c8710-409a-44ac-a5ab-fd57bb540034","Type":"ContainerStarted","Data":"1f25ba2bafc10e1e9cb4b55399e9fc23a4759f28a63fc2e37a007beb93db678b"}
Jul 24 17:56:32.477928 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:32.477564    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k" event={"ID":"186c8710-409a-44ac-a5ab-fd57bb540034","Type":"ContainerStarted","Data":"b24e9719344b00d762535092b2989abdaa34497189be91076b005823fcd47e80"}
Jul 24 17:56:32.477928 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:32.477590    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:32.495116 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:32.495072    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k" podStartSLOduration=2.495057598 podStartE2EDuration="2.495057598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:32.494393625 +0000 UTC m=+2781.465525805" watchObservedRunningTime="2026-07-24 17:56:32.495057598 +0000 UTC m=+2781.466189778"
Jul 24 17:56:43.485720 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:43.485691    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:56:49.221458 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.221426    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"]
Jul 24 17:56:49.224989 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.224966    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.227754 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.227732    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-bd0742-serving-cert\""
Jul 24 17:56:49.239686 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.239659    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"]
Jul 24 17:56:49.369892 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.369857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfrbw\" (UniqueName: \"kubernetes.io/projected/76520277-f9ec-40f8-b147-71dedc7f8516-kube-api-access-tfrbw\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.370113 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.369908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76520277-f9ec-40f8-b147-71dedc7f8516-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.370181 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.370122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76520277-f9ec-40f8-b147-71dedc7f8516-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.470751 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.470716    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76520277-f9ec-40f8-b147-71dedc7f8516-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.470751 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.470758    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76520277-f9ec-40f8-b147-71dedc7f8516-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.471062 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.470859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfrbw\" (UniqueName: \"kubernetes.io/projected/76520277-f9ec-40f8-b147-71dedc7f8516-kube-api-access-tfrbw\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.471452 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.471427    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76520277-f9ec-40f8-b147-71dedc7f8516-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.473667 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.473610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76520277-f9ec-40f8-b147-71dedc7f8516-maas-api-tls\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.479922 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.479891    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfrbw\" (UniqueName: \"kubernetes.io/projected/76520277-f9ec-40f8-b147-71dedc7f8516-kube-api-access-tfrbw\") pod \"maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.537860 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.537826    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:49.679505 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.679004    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"]
Jul 24 17:56:49.682012 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:56:49.681978    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76520277_f9ec_40f8_b147_71dedc7f8516.slice/crio-0ec884a7d5270a6eae2d4f31e8d19888a0f4568307f5029a81ece39b72a39e6f WatchSource:0}: Error finding container 0ec884a7d5270a6eae2d4f31e8d19888a0f4568307f5029a81ece39b72a39e6f: Status 404 returned error can't find the container with id 0ec884a7d5270a6eae2d4f31e8d19888a0f4568307f5029a81ece39b72a39e6f
Jul 24 17:56:49.683656 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.683636    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:56:49.899085 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.898982    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"]
Jul 24 17:56:49.903429 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.903415    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:49.906270 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.906249    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-bd0742-dockercfg-qprq8\""
Jul 24 17:56:49.906579 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.906563    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-bd0742\""
Jul 24 17:56:49.914563 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:49.914540    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"]
Jul 24 17:56:50.075877 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.075843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6\" (UID: \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:50.076093 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.075904    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7d6w\" (UniqueName: \"kubernetes.io/projected/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-kube-api-access-l7d6w\") pod \"payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6\" (UID: \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:50.176971 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.176877    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7d6w\" (UniqueName: \"kubernetes.io/projected/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-kube-api-access-l7d6w\") pod \"payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6\" (UID: \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:50.177172 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.176991    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6\" (UID: \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:50.177686 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.177668    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6\" (UID: \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:50.194719 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.194686    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7d6w\" (UniqueName: \"kubernetes.io/projected/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-kube-api-access-l7d6w\") pod \"payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6\" (UID: \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:50.214615 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.214587    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:50.352389 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.352328    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"]
Jul 24 17:56:50.354476 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:56:50.354449    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae6b699d_2c14_40f9_bbdb_a6dd6804c9ff.slice/crio-031364a768bb20996f75532b4a4c146e0d73cf564f9c1207a88b9606dae3e98d WatchSource:0}: Error finding container 031364a768bb20996f75532b4a4c146e0d73cf564f9c1207a88b9606dae3e98d: Status 404 returned error can't find the container with id 031364a768bb20996f75532b4a4c146e0d73cf564f9c1207a88b9606dae3e98d
Jul 24 17:56:50.562859 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.562822    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x" event={"ID":"76520277-f9ec-40f8-b147-71dedc7f8516","Type":"ContainerStarted","Data":"255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07"}
Jul 24 17:56:50.563080 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.562963    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:56:50.563080 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.563011    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x" event={"ID":"76520277-f9ec-40f8-b147-71dedc7f8516","Type":"ContainerStarted","Data":"0ec884a7d5270a6eae2d4f31e8d19888a0f4568307f5029a81ece39b72a39e6f"}
Jul 24 17:56:50.564240 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.564217    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6" event={"ID":"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff","Type":"ContainerStarted","Data":"8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb"}
Jul 24 17:56:50.564332 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.564262    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6" event={"ID":"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff","Type":"ContainerStarted","Data":"031364a768bb20996f75532b4a4c146e0d73cf564f9c1207a88b9606dae3e98d"}
Jul 24 17:56:50.564332 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.564291    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:56:50.580330 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.580282    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x" podStartSLOduration=1.274901774 podStartE2EDuration="1.580267352s" totalImagesPullingTime="305.365578ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:50.579514263 +0000 UTC m=+2799.550646445" watchObservedRunningTime="2026-07-24 17:56:50.580267352 +0000 UTC m=+2799.551399587"
Jul 24 17:56:50.596070 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:50.596004    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6" podStartSLOduration=1.595990222 podStartE2EDuration="1.595990222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:56:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:56:50.594939193 +0000 UTC m=+2799.566071375" watchObservedRunningTime="2026-07-24 17:56:50.595990222 +0000 UTC m=+2799.567122401"
Jul 24 17:56:56.575472 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:56:56.575443    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 17:57:01.576158 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:57:01.576120    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 17:58:06.739388 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.739313    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"]
Jul 24 17:58:06.742679 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.742662    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.745480 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.745457    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-ab19c5-serving-cert\""
Jul 24 17:58:06.763638 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.763613    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"]
Jul 24 17:58:06.828886 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.828837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0253d215-8bb3-49a2-8945-fbc2b330b0c8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.829112 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.828927    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccpd4\" (UniqueName: \"kubernetes.io/projected/0253d215-8bb3-49a2-8945-fbc2b330b0c8-kube-api-access-ccpd4\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.829112 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.829037    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0253d215-8bb3-49a2-8945-fbc2b330b0c8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.929625 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.929595    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0253d215-8bb3-49a2-8945-fbc2b330b0c8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.929814 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.929645    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccpd4\" (UniqueName: \"kubernetes.io/projected/0253d215-8bb3-49a2-8945-fbc2b330b0c8-kube-api-access-ccpd4\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.929814 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.929698    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0253d215-8bb3-49a2-8945-fbc2b330b0c8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.930327 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.930298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0253d215-8bb3-49a2-8945-fbc2b330b0c8-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.932343 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.932324    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0253d215-8bb3-49a2-8945-fbc2b330b0c8-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:06.939753 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:06.939729    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccpd4\" (UniqueName: \"kubernetes.io/projected/0253d215-8bb3-49a2-8945-fbc2b330b0c8-kube-api-access-ccpd4\") pod \"maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:07.053222 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.053161    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:07.206592 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.206568    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"]
Jul 24 17:58:07.208865 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:58:07.208837    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0253d215_8bb3_49a2_8945_fbc2b330b0c8.slice/crio-246e8345a9cfbe3f7637f3c48c8326afd3ef22aef992c7b42fc9a78ef4e5f35d WatchSource:0}: Error finding container 246e8345a9cfbe3f7637f3c48c8326afd3ef22aef992c7b42fc9a78ef4e5f35d: Status 404 returned error can't find the container with id 246e8345a9cfbe3f7637f3c48c8326afd3ef22aef992c7b42fc9a78ef4e5f35d
Jul 24 17:58:07.723035 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.723000    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"]
Jul 24 17:58:07.730771 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.730736    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:07.736689 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.736663    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ab19c5\""
Jul 24 17:58:07.737663 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.737494    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-ab19c5-dockercfg-xqtbd\""
Jul 24 17:58:07.740412 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.740387    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"]
Jul 24 17:58:07.798904 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.798862    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"]
Jul 24 17:58:07.802263 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.802248    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:07.823058 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.822999    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"]
Jul 24 17:58:07.837611 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.837576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndvkl\" (UniqueName: \"kubernetes.io/projected/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-kube-api-access-ndvkl\") pod \"payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr\" (UID: \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:07.837881 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.837855    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7fe025-938f-43de-be9e-dbce1431d810-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb\" (UID: \"ce7fe025-938f-43de-be9e-dbce1431d810\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:07.838085 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.838065    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frlqf\" (UniqueName: \"kubernetes.io/projected/ce7fe025-938f-43de-be9e-dbce1431d810-kube-api-access-frlqf\") pod \"payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb\" (UID: \"ce7fe025-938f-43de-be9e-dbce1431d810\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:07.838295 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.838278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr\" (UID: \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:07.917506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.917422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q" event={"ID":"0253d215-8bb3-49a2-8945-fbc2b330b0c8","Type":"ContainerStarted","Data":"6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea"}
Jul 24 17:58:07.917506 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.917484    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q" event={"ID":"0253d215-8bb3-49a2-8945-fbc2b330b0c8","Type":"ContainerStarted","Data":"246e8345a9cfbe3f7637f3c48c8326afd3ef22aef992c7b42fc9a78ef4e5f35d"}
Jul 24 17:58:07.917702 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.917619    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:07.939092 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.939003    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q" podStartSLOduration=1.499501024 podStartE2EDuration="1.938983577s" totalImagesPullingTime="439.482553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:58:07.936269621 +0000 UTC m=+2876.907401804" watchObservedRunningTime="2026-07-24 17:58:07.938983577 +0000 UTC m=+2876.910115759"
Jul 24 17:58:07.939280 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.939249    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr\" (UID: \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:07.939411 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.939389    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndvkl\" (UniqueName: \"kubernetes.io/projected/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-kube-api-access-ndvkl\") pod \"payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr\" (UID: \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:07.939540 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.939420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7fe025-938f-43de-be9e-dbce1431d810-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb\" (UID: \"ce7fe025-938f-43de-be9e-dbce1431d810\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:07.939540 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.939447    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frlqf\" (UniqueName: \"kubernetes.io/projected/ce7fe025-938f-43de-be9e-dbce1431d810-kube-api-access-frlqf\") pod \"payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb\" (UID: \"ce7fe025-938f-43de-be9e-dbce1431d810\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:07.940216 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.940193    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr\" (UID: \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:07.940301 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.940245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7fe025-938f-43de-be9e-dbce1431d810-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb\" (UID: \"ce7fe025-938f-43de-be9e-dbce1431d810\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:07.951122 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.951103    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndvkl\" (UniqueName: \"kubernetes.io/projected/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-kube-api-access-ndvkl\") pod \"payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr\" (UID: \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:07.951239 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:07.951220    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frlqf\" (UniqueName: \"kubernetes.io/projected/ce7fe025-938f-43de-be9e-dbce1431d810-kube-api-access-frlqf\") pod \"payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb\" (UID: \"ce7fe025-938f-43de-be9e-dbce1431d810\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:08.042435 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.042405    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:08.116497 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.116466    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:08.203778 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.203755    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"]
Jul 24 17:58:08.205919 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:58:08.205888    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a417d53_57f1_45a9_a3a4_5ab3f7aded7d.slice/crio-07f0c8009d5246f4be06b15535c2a3c9433a6585fc07e6029aefef645cb4cf19 WatchSource:0}: Error finding container 07f0c8009d5246f4be06b15535c2a3c9433a6585fc07e6029aefef645cb4cf19: Status 404 returned error can't find the container with id 07f0c8009d5246f4be06b15535c2a3c9433a6585fc07e6029aefef645cb4cf19
Jul 24 17:58:08.305003 ip-10-0-140-5 kubenswrapper[2562]: W0724 17:58:08.304974    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce7fe025_938f_43de_be9e_dbce1431d810.slice/crio-dccf984dd5dd0f3b8919e7652ddb858ae140966873655c38c8192af71f79287e WatchSource:0}: Error finding container dccf984dd5dd0f3b8919e7652ddb858ae140966873655c38c8192af71f79287e: Status 404 returned error can't find the container with id dccf984dd5dd0f3b8919e7652ddb858ae140966873655c38c8192af71f79287e
Jul 24 17:58:08.308917 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.308894    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"]
Jul 24 17:58:08.923881 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.923837    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb" event={"ID":"ce7fe025-938f-43de-be9e-dbce1431d810","Type":"ContainerStarted","Data":"48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab"}
Jul 24 17:58:08.924379 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.924008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:08.924379 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.924085    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb" event={"ID":"ce7fe025-938f-43de-be9e-dbce1431d810","Type":"ContainerStarted","Data":"dccf984dd5dd0f3b8919e7652ddb858ae140966873655c38c8192af71f79287e"}
Jul 24 17:58:08.925433 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.925402    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr" event={"ID":"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d","Type":"ContainerStarted","Data":"fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2"}
Jul 24 17:58:08.925552 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.925450    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr" event={"ID":"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d","Type":"ContainerStarted","Data":"07f0c8009d5246f4be06b15535c2a3c9433a6585fc07e6029aefef645cb4cf19"}
Jul 24 17:58:08.925607 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.925574    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:08.947393 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.947347    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb" podStartSLOduration=1.947333056 podStartE2EDuration="1.947333056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:58:08.945858157 +0000 UTC m=+2877.916990351" watchObservedRunningTime="2026-07-24 17:58:08.947333056 +0000 UTC m=+2877.918465237"
Jul 24 17:58:08.968491 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:08.968439    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr" podStartSLOduration=1.9684241230000001 podStartE2EDuration="1.968424123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:58:08.965451862 +0000 UTC m=+2877.936584034" watchObservedRunningTime="2026-07-24 17:58:08.968424123 +0000 UTC m=+2877.939556303"
Jul 24 17:58:13.930893 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:13.930865    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:19.934367 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:19.934317    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:19.934754 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:19.934541    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:32.421566 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.421532    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"]
Jul 24 17:58:32.422057 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.421755    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q" podUID="0253d215-8bb3-49a2-8945-fbc2b330b0c8" containerName="maas-api" containerID="cri-o://6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea" gracePeriod=30
Jul 24 17:58:32.474082 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.474056    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"]
Jul 24 17:58:32.474277 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.474251    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr" podUID="9a417d53-57f1-45a9-a3a4-5ab3f7aded7d" containerName="payload-processing" containerID="cri-o://fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2" gracePeriod=30
Jul 24 17:58:32.481933 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.481573    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"]
Jul 24 17:58:32.482519 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.482351    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb" podUID="ce7fe025-938f-43de-be9e-dbce1431d810" containerName="payload-pre-processing" containerID="cri-o://48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab" gracePeriod=30
Jul 24 17:58:32.782726 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.782700    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:32.785520 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.785494    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:32.797703 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.797687    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:32.961229 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961145    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-plugins-config-volume\") pod \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\" (UID: \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\") "
Jul 24 17:58:32.961229 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961186    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccpd4\" (UniqueName: \"kubernetes.io/projected/0253d215-8bb3-49a2-8945-fbc2b330b0c8-kube-api-access-ccpd4\") pod \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") "
Jul 24 17:58:32.961229 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961204    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7fe025-938f-43de-be9e-dbce1431d810-plugins-config-volume\") pod \"ce7fe025-938f-43de-be9e-dbce1431d810\" (UID: \"ce7fe025-938f-43de-be9e-dbce1431d810\") "
Jul 24 17:58:32.961516 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961247    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frlqf\" (UniqueName: \"kubernetes.io/projected/ce7fe025-938f-43de-be9e-dbce1431d810-kube-api-access-frlqf\") pod \"ce7fe025-938f-43de-be9e-dbce1431d810\" (UID: \"ce7fe025-938f-43de-be9e-dbce1431d810\") "
Jul 24 17:58:32.961516 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961284    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0253d215-8bb3-49a2-8945-fbc2b330b0c8-openshift-service-ca\") pod \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") "
Jul 24 17:58:32.961516 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961305    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0253d215-8bb3-49a2-8945-fbc2b330b0c8-maas-api-tls\") pod \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\" (UID: \"0253d215-8bb3-49a2-8945-fbc2b330b0c8\") "
Jul 24 17:58:32.961516 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961367    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndvkl\" (UniqueName: \"kubernetes.io/projected/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-kube-api-access-ndvkl\") pod \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\" (UID: \"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d\") "
Jul 24 17:58:32.961730 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961654    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0253d215-8bb3-49a2-8945-fbc2b330b0c8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0253d215-8bb3-49a2-8945-fbc2b330b0c8" (UID: "0253d215-8bb3-49a2-8945-fbc2b330b0c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:32.961730 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961671    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9a417d53-57f1-45a9-a3a4-5ab3f7aded7d" (UID: "9a417d53-57f1-45a9-a3a4-5ab3f7aded7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:32.961730 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.961681    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce7fe025-938f-43de-be9e-dbce1431d810-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ce7fe025-938f-43de-be9e-dbce1431d810" (UID: "ce7fe025-938f-43de-be9e-dbce1431d810"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:58:32.963533 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.963506    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7fe025-938f-43de-be9e-dbce1431d810-kube-api-access-frlqf" (OuterVolumeSpecName: "kube-api-access-frlqf") pod "ce7fe025-938f-43de-be9e-dbce1431d810" (UID: "ce7fe025-938f-43de-be9e-dbce1431d810"). InnerVolumeSpecName "kube-api-access-frlqf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:32.963652 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.963559    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0253d215-8bb3-49a2-8945-fbc2b330b0c8-kube-api-access-ccpd4" (OuterVolumeSpecName: "kube-api-access-ccpd4") pod "0253d215-8bb3-49a2-8945-fbc2b330b0c8" (UID: "0253d215-8bb3-49a2-8945-fbc2b330b0c8"). InnerVolumeSpecName "kube-api-access-ccpd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:32.963978 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.963950    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-kube-api-access-ndvkl" (OuterVolumeSpecName: "kube-api-access-ndvkl") pod "9a417d53-57f1-45a9-a3a4-5ab3f7aded7d" (UID: "9a417d53-57f1-45a9-a3a4-5ab3f7aded7d"). InnerVolumeSpecName "kube-api-access-ndvkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:58:32.964080 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:32.963955    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0253d215-8bb3-49a2-8945-fbc2b330b0c8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0253d215-8bb3-49a2-8945-fbc2b330b0c8" (UID: "0253d215-8bb3-49a2-8945-fbc2b330b0c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:58:33.047205 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.047176    2562 generic.go:356] "Generic (PLEG): container finished" podID="9a417d53-57f1-45a9-a3a4-5ab3f7aded7d" containerID="fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2" exitCode=0
Jul 24 17:58:33.047352 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.047217    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"
Jul 24 17:58:33.047352 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.047264    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr" event={"ID":"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d","Type":"ContainerDied","Data":"fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2"}
Jul 24 17:58:33.047352 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.047323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr" event={"ID":"9a417d53-57f1-45a9-a3a4-5ab3f7aded7d","Type":"ContainerDied","Data":"07f0c8009d5246f4be06b15535c2a3c9433a6585fc07e6029aefef645cb4cf19"}
Jul 24 17:58:33.047519 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.047353    2562 scope.go:117] "RemoveContainer" containerID="fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2"
Jul 24 17:58:33.048484 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.048451    2562 generic.go:356] "Generic (PLEG): container finished" podID="0253d215-8bb3-49a2-8945-fbc2b330b0c8" containerID="6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea" exitCode=0
Jul 24 17:58:33.048484 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.048481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q" event={"ID":"0253d215-8bb3-49a2-8945-fbc2b330b0c8","Type":"ContainerDied","Data":"6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea"}
Jul 24 17:58:33.048678 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.048503    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"
Jul 24 17:58:33.048678 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.048525    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q" event={"ID":"0253d215-8bb3-49a2-8945-fbc2b330b0c8","Type":"ContainerDied","Data":"246e8345a9cfbe3f7637f3c48c8326afd3ef22aef992c7b42fc9a78ef4e5f35d"}
Jul 24 17:58:33.050465 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.050420    2562 generic.go:356] "Generic (PLEG): container finished" podID="ce7fe025-938f-43de-be9e-dbce1431d810" containerID="48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab" exitCode=0
Jul 24 17:58:33.050465 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.050449    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"
Jul 24 17:58:33.050648 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.050505    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb" event={"ID":"ce7fe025-938f-43de-be9e-dbce1431d810","Type":"ContainerDied","Data":"48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab"}
Jul 24 17:58:33.050648 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.050552    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb" event={"ID":"ce7fe025-938f-43de-be9e-dbce1431d810","Type":"ContainerDied","Data":"dccf984dd5dd0f3b8919e7652ddb858ae140966873655c38c8192af71f79287e"}
Jul 24 17:58:33.056591 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.056573    2562 scope.go:117] "RemoveContainer" containerID="fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2"
Jul 24 17:58:33.056858 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:58:33.056838    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2\": container with ID starting with fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2 not found: ID does not exist" containerID="fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2"
Jul 24 17:58:33.056932 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.056864    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2"} err="failed to get container status \"fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2\": rpc error: code = NotFound desc = could not find container \"fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2\": container with ID starting with fc3aeb1cf2fd0226822c91f6a97f62a2093e8f1cb1ef9bca5a5d45c3d1262aa2 not found: ID does not exist"
Jul 24 17:58:33.056932 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.056879    2562 scope.go:117] "RemoveContainer" containerID="6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea"
Jul 24 17:58:33.062353 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.062320    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndvkl\" (UniqueName: \"kubernetes.io/projected/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-kube-api-access-ndvkl\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:58:33.062421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.062351    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:58:33.062421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.062368    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccpd4\" (UniqueName: \"kubernetes.io/projected/0253d215-8bb3-49a2-8945-fbc2b330b0c8-kube-api-access-ccpd4\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:58:33.062421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.062381    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce7fe025-938f-43de-be9e-dbce1431d810-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:58:33.062421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.062394    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frlqf\" (UniqueName: \"kubernetes.io/projected/ce7fe025-938f-43de-be9e-dbce1431d810-kube-api-access-frlqf\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:58:33.062421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.062407    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0253d215-8bb3-49a2-8945-fbc2b330b0c8-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:58:33.062421 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.062420    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0253d215-8bb3-49a2-8945-fbc2b330b0c8-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:58:33.065327 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.065312    2562 scope.go:117] "RemoveContainer" containerID="6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea"
Jul 24 17:58:33.065558 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:58:33.065543    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea\": container with ID starting with 6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea not found: ID does not exist" containerID="6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea"
Jul 24 17:58:33.065603 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.065563    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea"} err="failed to get container status \"6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea\": rpc error: code = NotFound desc = could not find container \"6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea\": container with ID starting with 6407b369a1d6aa932d43c5f14105953934ee73876f50b345e04573afd1d47fea not found: ID does not exist"
Jul 24 17:58:33.065603 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.065577    2562 scope.go:117] "RemoveContainer" containerID="48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab"
Jul 24 17:58:33.072663 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.072646    2562 scope.go:117] "RemoveContainer" containerID="48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab"
Jul 24 17:58:33.072982 ip-10-0-140-5 kubenswrapper[2562]: E0724 17:58:33.072941    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab\": container with ID starting with 48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab not found: ID does not exist" containerID="48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab"
Jul 24 17:58:33.072982 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.072967    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab"} err="failed to get container status \"48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab\": rpc error: code = NotFound desc = could not find container \"48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab\": container with ID starting with 48c94ca7a96540fec25b8eac8a477a4db2dfdcda3a31e067f0c8de0197340fab not found: ID does not exist"
Jul 24 17:58:33.075167 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.075141    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"]
Jul 24 17:58:33.077199 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.077179    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ab19c5-6b879847cd-lpvqr"]
Jul 24 17:58:33.088361 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.088341    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"]
Jul 24 17:58:33.091038 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.090995    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ab19c5-7d6658b59d-5sh7q"]
Jul 24 17:58:33.102676 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.102657    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"]
Jul 24 17:58:33.108967 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.108949    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ab19c5-d56bbdcdd-cb2tb"]
Jul 24 17:58:33.506162 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.506138    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0253d215-8bb3-49a2-8945-fbc2b330b0c8" path="/var/lib/kubelet/pods/0253d215-8bb3-49a2-8945-fbc2b330b0c8/volumes"
Jul 24 17:58:33.506505 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.506476    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a417d53-57f1-45a9-a3a4-5ab3f7aded7d" path="/var/lib/kubelet/pods/9a417d53-57f1-45a9-a3a4-5ab3f7aded7d/volumes"
Jul 24 17:58:33.506772 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:58:33.506760    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce7fe025-938f-43de-be9e-dbce1431d810" path="/var/lib/kubelet/pods/ce7fe025-938f-43de-be9e-dbce1431d810/volumes"
Jul 24 17:59:02.901930 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:02.901899    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"]
Jul 24 17:59:02.902524 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:02.902139    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k" podUID="186c8710-409a-44ac-a5ab-fd57bb540034" containerName="payload-pre-processing" containerID="cri-o://1f25ba2bafc10e1e9cb4b55399e9fc23a4759f28a63fc2e37a007beb93db678b" gracePeriod=30
Jul 24 17:59:03.191158 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.191127    2562 generic.go:356] "Generic (PLEG): container finished" podID="186c8710-409a-44ac-a5ab-fd57bb540034" containerID="1f25ba2bafc10e1e9cb4b55399e9fc23a4759f28a63fc2e37a007beb93db678b" exitCode=0
Jul 24 17:59:03.191304 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.191198    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k" event={"ID":"186c8710-409a-44ac-a5ab-fd57bb540034","Type":"ContainerDied","Data":"1f25ba2bafc10e1e9cb4b55399e9fc23a4759f28a63fc2e37a007beb93db678b"}
Jul 24 17:59:03.191304 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.191248    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k" event={"ID":"186c8710-409a-44ac-a5ab-fd57bb540034","Type":"ContainerDied","Data":"b24e9719344b00d762535092b2989abdaa34497189be91076b005823fcd47e80"}
Jul 24 17:59:03.191304 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.191265    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b24e9719344b00d762535092b2989abdaa34497189be91076b005823fcd47e80"
Jul 24 17:59:03.198470 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.198445    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:59:03.321362 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.321335    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/186c8710-409a-44ac-a5ab-fd57bb540034-plugins-config-volume\") pod \"186c8710-409a-44ac-a5ab-fd57bb540034\" (UID: \"186c8710-409a-44ac-a5ab-fd57bb540034\") "
Jul 24 17:59:03.321553 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.321395    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2zq8\" (UniqueName: \"kubernetes.io/projected/186c8710-409a-44ac-a5ab-fd57bb540034-kube-api-access-w2zq8\") pod \"186c8710-409a-44ac-a5ab-fd57bb540034\" (UID: \"186c8710-409a-44ac-a5ab-fd57bb540034\") "
Jul 24 17:59:03.321753 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.321728    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/186c8710-409a-44ac-a5ab-fd57bb540034-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "186c8710-409a-44ac-a5ab-fd57bb540034" (UID: "186c8710-409a-44ac-a5ab-fd57bb540034"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:59:03.323624 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.323592    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186c8710-409a-44ac-a5ab-fd57bb540034-kube-api-access-w2zq8" (OuterVolumeSpecName: "kube-api-access-w2zq8") pod "186c8710-409a-44ac-a5ab-fd57bb540034" (UID: "186c8710-409a-44ac-a5ab-fd57bb540034"). InnerVolumeSpecName "kube-api-access-w2zq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:59:03.422387 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.422359    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/186c8710-409a-44ac-a5ab-fd57bb540034-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:59:03.422387 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:03.422382    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2zq8\" (UniqueName: \"kubernetes.io/projected/186c8710-409a-44ac-a5ab-fd57bb540034-kube-api-access-w2zq8\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 17:59:04.195963 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:04.195931    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"
Jul 24 17:59:04.233785 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:04.233748    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"]
Jul 24 17:59:04.252767 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:04.252735    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a6414-8645995fc6-m8n6k"]
Jul 24 17:59:05.505162 ip-10-0-140-5 kubenswrapper[2562]: I0724 17:59:05.505130    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186c8710-409a-44ac-a5ab-fd57bb540034" path="/var/lib/kubelet/pods/186c8710-409a-44ac-a5ab-fd57bb540034/volumes"
Jul 24 18:00:07.393156 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.393051    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"]
Jul 24 18:00:07.393767 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.393283    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x" podUID="76520277-f9ec-40f8-b147-71dedc7f8516" containerName="maas-api" containerID="cri-o://255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07" gracePeriod=30
Jul 24 18:00:07.475947 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.475909    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"]
Jul 24 18:00:07.476555 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.476513    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6" podUID="ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff" containerName="payload-pre-processing" containerID="cri-o://8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb" gracePeriod=30
Jul 24 18:00:07.753343 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.753320    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 18:00:07.756411 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.756393    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 18:00:07.872178 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.872143    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76520277-f9ec-40f8-b147-71dedc7f8516-maas-api-tls\") pod \"76520277-f9ec-40f8-b147-71dedc7f8516\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") "
Jul 24 18:00:07.872178 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.872195    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76520277-f9ec-40f8-b147-71dedc7f8516-openshift-service-ca\") pod \"76520277-f9ec-40f8-b147-71dedc7f8516\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") "
Jul 24 18:00:07.872426 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.872228    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-plugins-config-volume\") pod \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\" (UID: \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\") "
Jul 24 18:00:07.872426 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.872256    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7d6w\" (UniqueName: \"kubernetes.io/projected/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-kube-api-access-l7d6w\") pod \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\" (UID: \"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff\") "
Jul 24 18:00:07.872426 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.872324    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfrbw\" (UniqueName: \"kubernetes.io/projected/76520277-f9ec-40f8-b147-71dedc7f8516-kube-api-access-tfrbw\") pod \"76520277-f9ec-40f8-b147-71dedc7f8516\" (UID: \"76520277-f9ec-40f8-b147-71dedc7f8516\") "
Jul 24 18:00:07.872603 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.872563    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76520277-f9ec-40f8-b147-71dedc7f8516-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "76520277-f9ec-40f8-b147-71dedc7f8516" (UID: "76520277-f9ec-40f8-b147-71dedc7f8516"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:07.872715 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.872690    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff" (UID: "ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:07.874413 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.874393    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76520277-f9ec-40f8-b147-71dedc7f8516-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "76520277-f9ec-40f8-b147-71dedc7f8516" (UID: "76520277-f9ec-40f8-b147-71dedc7f8516"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:00:07.874506 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.874476    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-kube-api-access-l7d6w" (OuterVolumeSpecName: "kube-api-access-l7d6w") pod "ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff" (UID: "ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff"). InnerVolumeSpecName "kube-api-access-l7d6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:07.874506 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.874483    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76520277-f9ec-40f8-b147-71dedc7f8516-kube-api-access-tfrbw" (OuterVolumeSpecName: "kube-api-access-tfrbw") pod "76520277-f9ec-40f8-b147-71dedc7f8516" (UID: "76520277-f9ec-40f8-b147-71dedc7f8516"). InnerVolumeSpecName "kube-api-access-tfrbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:07.973265 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.973239    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:07.973265 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.973262    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7d6w\" (UniqueName: \"kubernetes.io/projected/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff-kube-api-access-l7d6w\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:07.973432 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.973273    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfrbw\" (UniqueName: \"kubernetes.io/projected/76520277-f9ec-40f8-b147-71dedc7f8516-kube-api-access-tfrbw\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:07.973432 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.973282    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76520277-f9ec-40f8-b147-71dedc7f8516-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:07.973432 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:07.973291    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76520277-f9ec-40f8-b147-71dedc7f8516-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:08.502084 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.502049    2562 generic.go:356] "Generic (PLEG): container finished" podID="76520277-f9ec-40f8-b147-71dedc7f8516" containerID="255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07" exitCode=0
Jul 24 18:00:08.502480 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.502098    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"
Jul 24 18:00:08.502480 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.502133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x" event={"ID":"76520277-f9ec-40f8-b147-71dedc7f8516","Type":"ContainerDied","Data":"255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07"}
Jul 24 18:00:08.502480 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.502174    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x" event={"ID":"76520277-f9ec-40f8-b147-71dedc7f8516","Type":"ContainerDied","Data":"0ec884a7d5270a6eae2d4f31e8d19888a0f4568307f5029a81ece39b72a39e6f"}
Jul 24 18:00:08.502480 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.502204    2562 scope.go:117] "RemoveContainer" containerID="255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07"
Jul 24 18:00:08.503497 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.503471    2562 generic.go:356] "Generic (PLEG): container finished" podID="ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff" containerID="8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb" exitCode=0
Jul 24 18:00:08.503586 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.503500    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6" event={"ID":"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff","Type":"ContainerDied","Data":"8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb"}
Jul 24 18:00:08.503586 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.503547    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6" event={"ID":"ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff","Type":"ContainerDied","Data":"031364a768bb20996f75532b4a4c146e0d73cf564f9c1207a88b9606dae3e98d"}
Jul 24 18:00:08.503586 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.503508    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"
Jul 24 18:00:08.513699 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.513683    2562 scope.go:117] "RemoveContainer" containerID="255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07"
Jul 24 18:00:08.513956 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:08.513936    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07\": container with ID starting with 255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07 not found: ID does not exist" containerID="255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07"
Jul 24 18:00:08.514065 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.513965    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07"} err="failed to get container status \"255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07\": rpc error: code = NotFound desc = could not find container \"255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07\": container with ID starting with 255c69d275dca04892e779f9178515e37a2aab238ec54f81da15c662438fad07 not found: ID does not exist"
Jul 24 18:00:08.514065 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.513982    2562 scope.go:117] "RemoveContainer" containerID="8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb"
Jul 24 18:00:08.521954 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.521937    2562 scope.go:117] "RemoveContainer" containerID="8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb"
Jul 24 18:00:08.522249 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:08.522230    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb\": container with ID starting with 8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb not found: ID does not exist" containerID="8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb"
Jul 24 18:00:08.522325 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.522258    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb"} err="failed to get container status \"8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb\": rpc error: code = NotFound desc = could not find container \"8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb\": container with ID starting with 8807c5458ce2945dc2e6c8bf74eb500ace8adb6f4cc4b2202bce684e7e26c7bb not found: ID does not exist"
Jul 24 18:00:08.528566 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.528543    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"]
Jul 24 18:00:08.531873 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.531854    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-bd0742-74c49db8d4-h92s6"]
Jul 24 18:00:08.542555 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.542532    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"]
Jul 24 18:00:08.549327 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:08.549309    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-bd0742-9c48d88b8-mkn2x"]
Jul 24 18:00:09.506000 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:09.505967    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76520277-f9ec-40f8-b147-71dedc7f8516" path="/var/lib/kubelet/pods/76520277-f9ec-40f8-b147-71dedc7f8516/volumes"
Jul 24 18:00:09.506478 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:09.506461    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff" path="/var/lib/kubelet/pods/ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff/volumes"
Jul 24 18:00:11.635264 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:11.635151    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 18:00:11.652340 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:11.642468    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zp6nk_697f9852-7423-45f4-851e-89da6d7e593f/ovn-acl-logging/0.log"
Jul 24 18:00:29.736954 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:29.736115    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"]
Jul 24 18:00:29.736954 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:29.736366    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv" podUID="1dfa0922-d592-48aa-adff-e3a542c54e5b" containerName="maas-api" containerID="cri-o://27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e" gracePeriod=30
Jul 24 18:00:29.786811 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:29.786778    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"]
Jul 24 18:00:29.786991 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:29.786966    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk" podUID="a2519ca4-8872-4038-bb16-f2da6f087073" containerName="payload-processing" containerID="cri-o://1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9" gracePeriod=30
Jul 24 18:00:29.800055 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:29.799999    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"]
Jul 24 18:00:29.800312 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:29.800266    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j" podUID="d971647a-5331-4212-b720-2f5f90243930" containerName="payload-pre-processing" containerID="cri-o://cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559" gracePeriod=30
Jul 24 18:00:30.063619 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.063591    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 18:00:30.095047 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.095008    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 18:00:30.127784 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.127760    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 18:00:30.161138 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161112    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfa0922-d592-48aa-adff-e3a542c54e5b-maas-api-tls\") pod \"1dfa0922-d592-48aa-adff-e3a542c54e5b\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") "
Jul 24 18:00:30.161330 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161151    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfa0922-d592-48aa-adff-e3a542c54e5b-openshift-service-ca\") pod \"1dfa0922-d592-48aa-adff-e3a542c54e5b\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") "
Jul 24 18:00:30.161330 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161192    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d971647a-5331-4212-b720-2f5f90243930-plugins-config-volume\") pod \"d971647a-5331-4212-b720-2f5f90243930\" (UID: \"d971647a-5331-4212-b720-2f5f90243930\") "
Jul 24 18:00:30.161330 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161246    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2519ca4-8872-4038-bb16-f2da6f087073-plugins-config-volume\") pod \"a2519ca4-8872-4038-bb16-f2da6f087073\" (UID: \"a2519ca4-8872-4038-bb16-f2da6f087073\") "
Jul 24 18:00:30.161330 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161273    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpkzp\" (UniqueName: \"kubernetes.io/projected/a2519ca4-8872-4038-bb16-f2da6f087073-kube-api-access-gpkzp\") pod \"a2519ca4-8872-4038-bb16-f2da6f087073\" (UID: \"a2519ca4-8872-4038-bb16-f2da6f087073\") "
Jul 24 18:00:30.161330 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161328    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2fbn\" (UniqueName: \"kubernetes.io/projected/d971647a-5331-4212-b720-2f5f90243930-kube-api-access-x2fbn\") pod \"d971647a-5331-4212-b720-2f5f90243930\" (UID: \"d971647a-5331-4212-b720-2f5f90243930\") "
Jul 24 18:00:30.161591 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161365    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjkbj\" (UniqueName: \"kubernetes.io/projected/1dfa0922-d592-48aa-adff-e3a542c54e5b-kube-api-access-cjkbj\") pod \"1dfa0922-d592-48aa-adff-e3a542c54e5b\" (UID: \"1dfa0922-d592-48aa-adff-e3a542c54e5b\") "
Jul 24 18:00:30.161643 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161605    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dfa0922-d592-48aa-adff-e3a542c54e5b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1dfa0922-d592-48aa-adff-e3a542c54e5b" (UID: "1dfa0922-d592-48aa-adff-e3a542c54e5b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:30.161643 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161607    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d971647a-5331-4212-b720-2f5f90243930-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d971647a-5331-4212-b720-2f5f90243930" (UID: "d971647a-5331-4212-b720-2f5f90243930"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:30.161733 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.161642    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2519ca4-8872-4038-bb16-f2da6f087073-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2519ca4-8872-4038-bb16-f2da6f087073" (UID: "a2519ca4-8872-4038-bb16-f2da6f087073"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:30.163514 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.163488    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dfa0922-d592-48aa-adff-e3a542c54e5b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1dfa0922-d592-48aa-adff-e3a542c54e5b" (UID: "1dfa0922-d592-48aa-adff-e3a542c54e5b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:00:30.163667 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.163645    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dfa0922-d592-48aa-adff-e3a542c54e5b-kube-api-access-cjkbj" (OuterVolumeSpecName: "kube-api-access-cjkbj") pod "1dfa0922-d592-48aa-adff-e3a542c54e5b" (UID: "1dfa0922-d592-48aa-adff-e3a542c54e5b"). InnerVolumeSpecName "kube-api-access-cjkbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:30.163717 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.163685    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2519ca4-8872-4038-bb16-f2da6f087073-kube-api-access-gpkzp" (OuterVolumeSpecName: "kube-api-access-gpkzp") pod "a2519ca4-8872-4038-bb16-f2da6f087073" (UID: "a2519ca4-8872-4038-bb16-f2da6f087073"). InnerVolumeSpecName "kube-api-access-gpkzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:30.163887 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.163869    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d971647a-5331-4212-b720-2f5f90243930-kube-api-access-x2fbn" (OuterVolumeSpecName: "kube-api-access-x2fbn") pod "d971647a-5331-4212-b720-2f5f90243930" (UID: "d971647a-5331-4212-b720-2f5f90243930"). InnerVolumeSpecName "kube-api-access-x2fbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:30.261936 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.261909    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2fbn\" (UniqueName: \"kubernetes.io/projected/d971647a-5331-4212-b720-2f5f90243930-kube-api-access-x2fbn\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:30.261936 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.261930    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjkbj\" (UniqueName: \"kubernetes.io/projected/1dfa0922-d592-48aa-adff-e3a542c54e5b-kube-api-access-cjkbj\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:30.261936 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.261940    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1dfa0922-d592-48aa-adff-e3a542c54e5b-maas-api-tls\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:30.262164 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.261949    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1dfa0922-d592-48aa-adff-e3a542c54e5b-openshift-service-ca\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:30.262164 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.261958    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d971647a-5331-4212-b720-2f5f90243930-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:30.262164 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.261966    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2519ca4-8872-4038-bb16-f2da6f087073-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:30.262164 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.261975    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpkzp\" (UniqueName: \"kubernetes.io/projected/a2519ca4-8872-4038-bb16-f2da6f087073-kube-api-access-gpkzp\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:30.608555 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.608514    2562 generic.go:356] "Generic (PLEG): container finished" podID="1dfa0922-d592-48aa-adff-e3a542c54e5b" containerID="27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e" exitCode=0
Jul 24 18:00:30.608739 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.608574    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"
Jul 24 18:00:30.608739 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.608602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv" event={"ID":"1dfa0922-d592-48aa-adff-e3a542c54e5b","Type":"ContainerDied","Data":"27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e"}
Jul 24 18:00:30.608739 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.608670    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv" event={"ID":"1dfa0922-d592-48aa-adff-e3a542c54e5b","Type":"ContainerDied","Data":"d313bba8736042cbfd2adc281e536bc593c87d691ad22e5fa4b8b70a18835242"}
Jul 24 18:00:30.608739 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.608705    2562 scope.go:117] "RemoveContainer" containerID="27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e"
Jul 24 18:00:30.610333 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.610308    2562 generic.go:356] "Generic (PLEG): container finished" podID="a2519ca4-8872-4038-bb16-f2da6f087073" containerID="1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9" exitCode=0
Jul 24 18:00:30.610469 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.610343    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"
Jul 24 18:00:30.610469 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.610410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk" event={"ID":"a2519ca4-8872-4038-bb16-f2da6f087073","Type":"ContainerDied","Data":"1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9"}
Jul 24 18:00:30.610469 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.610452    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk" event={"ID":"a2519ca4-8872-4038-bb16-f2da6f087073","Type":"ContainerDied","Data":"5fe3df115750b63e2a10e4ba2722620311e4fe9125b23c0ed8f9060001bf2509"}
Jul 24 18:00:30.611836 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.611812    2562 generic.go:356] "Generic (PLEG): container finished" podID="d971647a-5331-4212-b720-2f5f90243930" containerID="cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559" exitCode=0
Jul 24 18:00:30.611946 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.611869    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"
Jul 24 18:00:30.611946 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.611898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j" event={"ID":"d971647a-5331-4212-b720-2f5f90243930","Type":"ContainerDied","Data":"cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559"}
Jul 24 18:00:30.612051 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.611955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j" event={"ID":"d971647a-5331-4212-b720-2f5f90243930","Type":"ContainerDied","Data":"20d63963457984545ed31d24f202d5a6ad47785fd95230a685030f20f5414d02"}
Jul 24 18:00:30.618692 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.618663    2562 scope.go:117] "RemoveContainer" containerID="27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e"
Jul 24 18:00:30.618963 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:30.618941    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e\": container with ID starting with 27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e not found: ID does not exist" containerID="27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e"
Jul 24 18:00:30.619146 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.618969    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e"} err="failed to get container status \"27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e\": rpc error: code = NotFound desc = could not find container \"27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e\": container with ID starting with 27ea8b4be64d8e2aed3f0f5a2d3a5e18c29f6796ce2e6e5bc0c6786f201e996e not found: ID does not exist"
Jul 24 18:00:30.619146 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.618985    2562 scope.go:117] "RemoveContainer" containerID="1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9"
Jul 24 18:00:30.627638 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.627618    2562 scope.go:117] "RemoveContainer" containerID="1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9"
Jul 24 18:00:30.627857 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:30.627841    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9\": container with ID starting with 1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9 not found: ID does not exist" containerID="1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9"
Jul 24 18:00:30.627913 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.627865    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9"} err="failed to get container status \"1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9\": rpc error: code = NotFound desc = could not find container \"1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9\": container with ID starting with 1deb55d295bde148b367c23b39343eee3814025238f36bd77102ec719ddc18c9 not found: ID does not exist"
Jul 24 18:00:30.627913 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.627881    2562 scope.go:117] "RemoveContainer" containerID="cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559"
Jul 24 18:00:30.632459 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.632441    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"]
Jul 24 18:00:30.634955 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.634935    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bb94b3-67c58c7b74-h6wtv"]
Jul 24 18:00:30.636003 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.635988    2562 scope.go:117] "RemoveContainer" containerID="cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559"
Jul 24 18:00:30.636283 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:30.636264    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559\": container with ID starting with cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559 not found: ID does not exist" containerID="cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559"
Jul 24 18:00:30.636331 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.636291    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559"} err="failed to get container status \"cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559\": rpc error: code = NotFound desc = could not find container \"cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559\": container with ID starting with cee6e6a1d079f0e71ed6b772971d1dda423baa8f6f0b2a8ddb7eac8fd53b7559 not found: ID does not exist"
Jul 24 18:00:30.645663 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.645641    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"]
Jul 24 18:00:30.650551 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.650529    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bb94b3-649846698b-j69tk"]
Jul 24 18:00:30.660560 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.660541    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"]
Jul 24 18:00:30.667249 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:30.667230    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bb94b3-6bbd75db56-f6s9j"]
Jul 24 18:00:31.505562 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:31.505529    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dfa0922-d592-48aa-adff-e3a542c54e5b" path="/var/lib/kubelet/pods/1dfa0922-d592-48aa-adff-e3a542c54e5b/volumes"
Jul 24 18:00:31.506074 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:31.506051    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2519ca4-8872-4038-bb16-f2da6f087073" path="/var/lib/kubelet/pods/a2519ca4-8872-4038-bb16-f2da6f087073/volumes"
Jul 24 18:00:31.506437 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:31.506418    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d971647a-5331-4212-b720-2f5f90243930" path="/var/lib/kubelet/pods/d971647a-5331-4212-b720-2f5f90243930/volumes"
Jul 24 18:00:41.192319 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:41.192278    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"]
Jul 24 18:00:41.193274 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:41.192534    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy" containerID="cri-o://e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c" gracePeriod=30
Jul 24 18:00:45.388401 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:45.388357    2562 patch_prober.go:28] interesting pod/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 18:00:45.388809 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:45.388422    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 18:00:46.448059 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.448034    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 18:00:46.498003 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.497942    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c95dd0-732f-4362-a869-d2484c966208-istio-podinfo\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498003 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.497972    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nxns\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-kube-api-access-4nxns\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498197 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498043    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c95dd0-732f-4362-a869-d2484c966208-istiod-ca-cert\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498197 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498079    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-socket\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498197 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498125    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-data\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498335 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498196    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-credential-socket\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498335 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498241    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-certs\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498335 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498264    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-istio-token\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498335 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498299    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-envoy\") pod \"98c95dd0-732f-4362-a869-d2484c966208\" (UID: \"98c95dd0-732f-4362-a869-d2484c966208\") "
Jul 24 18:00:46.498539 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498489    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:46.498539 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498506    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:46.498656 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498617    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.498656 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498635    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.498999 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.498978    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:46.499205 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.499155    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-data" (OuterVolumeSpecName: "istio-data") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:46.499205 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.499175    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98c95dd0-732f-4362-a869-d2484c966208-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:46.500363 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.500335    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-kube-api-access-4nxns" (OuterVolumeSpecName: "kube-api-access-4nxns") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "kube-api-access-4nxns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:46.500723 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.500699    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98c95dd0-732f-4362-a869-d2484c966208-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 18:00:46.500975 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.500948    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-istio-token" (OuterVolumeSpecName: "istio-token") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:46.501096 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.500953    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "98c95dd0-732f-4362-a869-d2484c966208" (UID: "98c95dd0-732f-4362-a869-d2484c966208"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:00:46.599252 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.599223    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.599252 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.599249    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.599425 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.599262    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c95dd0-732f-4362-a869-d2484c966208-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.599425 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.599274    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nxns\" (UniqueName: \"kubernetes.io/projected/98c95dd0-732f-4362-a869-d2484c966208-kube-api-access-4nxns\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.599425 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.599282    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c95dd0-732f-4362-a869-d2484c966208-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.599425 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.599290    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.599425 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.599297    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c95dd0-732f-4362-a869-d2484c966208-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:46.690780 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.690747    2562 generic.go:356] "Generic (PLEG): container finished" podID="98c95dd0-732f-4362-a869-d2484c966208" containerID="e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c" exitCode=0
Jul 24 18:00:46.690934 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.690807    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"
Jul 24 18:00:46.690934 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.690829    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" event={"ID":"98c95dd0-732f-4362-a869-d2484c966208","Type":"ContainerDied","Data":"e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c"}
Jul 24 18:00:46.690934 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.690882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f" event={"ID":"98c95dd0-732f-4362-a869-d2484c966208","Type":"ContainerDied","Data":"5e69eac713ae7b10ff7299b676e40184e6ae3b19db715dbf9674ae12daa2fe50"}
Jul 24 18:00:46.690934 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.690903    2562 scope.go:117] "RemoveContainer" containerID="e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c"
Jul 24 18:00:46.699815 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.699798    2562 scope.go:117] "RemoveContainer" containerID="743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6"
Jul 24 18:00:46.707503 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.707489    2562 scope.go:117] "RemoveContainer" containerID="e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c"
Jul 24 18:00:46.707777 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:46.707757    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c\": container with ID starting with e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c not found: ID does not exist" containerID="e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c"
Jul 24 18:00:46.707879 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.707782    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c"} err="failed to get container status \"e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c\": rpc error: code = NotFound desc = could not find container \"e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c\": container with ID starting with e3d32aaa5c4e9579575cafb5b9ff4fba0c819cb83290af63e0f5428d607d837c not found: ID does not exist"
Jul 24 18:00:46.707879 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.707797    2562 scope.go:117] "RemoveContainer" containerID="743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6"
Jul 24 18:00:46.708073 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:46.708055    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6\": container with ID starting with 743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6 not found: ID does not exist" containerID="743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6"
Jul 24 18:00:46.708141 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.708086    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6"} err="failed to get container status \"743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6\": rpc error: code = NotFound desc = could not find container \"743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6\": container with ID starting with 743faf73b1184b4d81706f3e521ebe1978cffbf70cf1c9e5cad3b7f6d3eebce6 not found: ID does not exist"
Jul 24 18:00:46.713423 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.713402    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"]
Jul 24 18:00:46.717662 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:46.717639    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-bb94b3-openshift-default-744f4c755b-h4v9f"]
Jul 24 18:00:47.505268 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:47.505241    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c95dd0-732f-4362-a869-d2484c966208" path="/var/lib/kubelet/pods/98c95dd0-732f-4362-a869-d2484c966208/volumes"
Jul 24 18:00:52.038166 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.038120    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"]
Jul 24 18:00:52.038703 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.038382    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl" podUID="e3253f25-a9d7-4364-9b92-21e17d22e9f3" containerName="payload-pre-processing" containerID="cri-o://3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec" gracePeriod=30
Jul 24 18:00:52.039920 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.039879    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"]
Jul 24 18:00:52.040138 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.040095    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8" podUID="503b5993-5781-46b0-8ea5-c328eb0905be" containerName="payload-processing" containerID="cri-o://77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f" gracePeriod=30
Jul 24 18:00:52.317393 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.317372    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 18:00:52.320116 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.320097    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 18:00:52.444312 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.444285    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ng82\" (UniqueName: \"kubernetes.io/projected/e3253f25-a9d7-4364-9b92-21e17d22e9f3-kube-api-access-6ng82\") pod \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\" (UID: \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\") "
Jul 24 18:00:52.444490 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.444359    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3253f25-a9d7-4364-9b92-21e17d22e9f3-plugins-config-volume\") pod \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\" (UID: \"e3253f25-a9d7-4364-9b92-21e17d22e9f3\") "
Jul 24 18:00:52.444490 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.444456    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503b5993-5781-46b0-8ea5-c328eb0905be-plugins-config-volume\") pod \"503b5993-5781-46b0-8ea5-c328eb0905be\" (UID: \"503b5993-5781-46b0-8ea5-c328eb0905be\") "
Jul 24 18:00:52.444622 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.444491    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzjh4\" (UniqueName: \"kubernetes.io/projected/503b5993-5781-46b0-8ea5-c328eb0905be-kube-api-access-qzjh4\") pod \"503b5993-5781-46b0-8ea5-c328eb0905be\" (UID: \"503b5993-5781-46b0-8ea5-c328eb0905be\") "
Jul 24 18:00:52.444841 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.444810    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3253f25-a9d7-4364-9b92-21e17d22e9f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3253f25-a9d7-4364-9b92-21e17d22e9f3" (UID: "e3253f25-a9d7-4364-9b92-21e17d22e9f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:52.444841 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.444829    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/503b5993-5781-46b0-8ea5-c328eb0905be-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "503b5993-5781-46b0-8ea5-c328eb0905be" (UID: "503b5993-5781-46b0-8ea5-c328eb0905be"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:00:52.446529 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.446510    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3253f25-a9d7-4364-9b92-21e17d22e9f3-kube-api-access-6ng82" (OuterVolumeSpecName: "kube-api-access-6ng82") pod "e3253f25-a9d7-4364-9b92-21e17d22e9f3" (UID: "e3253f25-a9d7-4364-9b92-21e17d22e9f3"). InnerVolumeSpecName "kube-api-access-6ng82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:52.446732 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.446714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/503b5993-5781-46b0-8ea5-c328eb0905be-kube-api-access-qzjh4" (OuterVolumeSpecName: "kube-api-access-qzjh4") pod "503b5993-5781-46b0-8ea5-c328eb0905be" (UID: "503b5993-5781-46b0-8ea5-c328eb0905be"). InnerVolumeSpecName "kube-api-access-qzjh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:00:52.546136 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.546096    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ng82\" (UniqueName: \"kubernetes.io/projected/e3253f25-a9d7-4364-9b92-21e17d22e9f3-kube-api-access-6ng82\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:52.546136 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.546130    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3253f25-a9d7-4364-9b92-21e17d22e9f3-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:52.546341 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.546146    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/503b5993-5781-46b0-8ea5-c328eb0905be-plugins-config-volume\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:52.546341 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.546161    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzjh4\" (UniqueName: \"kubernetes.io/projected/503b5993-5781-46b0-8ea5-c328eb0905be-kube-api-access-qzjh4\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:00:52.718323 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.718294    2562 generic.go:356] "Generic (PLEG): container finished" podID="e3253f25-a9d7-4364-9b92-21e17d22e9f3" containerID="3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec" exitCode=0
Jul 24 18:00:52.718489 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.718343    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"
Jul 24 18:00:52.718489 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.718375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl" event={"ID":"e3253f25-a9d7-4364-9b92-21e17d22e9f3","Type":"ContainerDied","Data":"3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec"}
Jul 24 18:00:52.718489 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.718429    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl" event={"ID":"e3253f25-a9d7-4364-9b92-21e17d22e9f3","Type":"ContainerDied","Data":"bec34a8e1d68e974dad41b0b34689db2e8f616ec0090fceb523646e523921070"}
Jul 24 18:00:52.718489 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.718453    2562 scope.go:117] "RemoveContainer" containerID="3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec"
Jul 24 18:00:52.719645 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.719628    2562 generic.go:356] "Generic (PLEG): container finished" podID="503b5993-5781-46b0-8ea5-c328eb0905be" containerID="77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f" exitCode=0
Jul 24 18:00:52.719724 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.719665    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"
Jul 24 18:00:52.719724 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.719696    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8" event={"ID":"503b5993-5781-46b0-8ea5-c328eb0905be","Type":"ContainerDied","Data":"77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f"}
Jul 24 18:00:52.719833 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.719728    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8" event={"ID":"503b5993-5781-46b0-8ea5-c328eb0905be","Type":"ContainerDied","Data":"4b6c26cffca710e3fd5e4b562c71fc01739bb4dc4e315e10089f6dee130f5a8b"}
Jul 24 18:00:52.727719 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.727703    2562 scope.go:117] "RemoveContainer" containerID="3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec"
Jul 24 18:00:52.728002 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:52.727977    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec\": container with ID starting with 3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec not found: ID does not exist" containerID="3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec"
Jul 24 18:00:52.728128 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.728006    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec"} err="failed to get container status \"3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec\": rpc error: code = NotFound desc = could not find container \"3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec\": container with ID starting with 3654a7d13fba6afe8002ea5311804a25ff855c308456c0bf92913adc429a8dec not found: ID does not exist"
Jul 24 18:00:52.728128 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.728049    2562 scope.go:117] "RemoveContainer" containerID="77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f"
Jul 24 18:00:52.736203 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.736187    2562 scope.go:117] "RemoveContainer" containerID="77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f"
Jul 24 18:00:52.736428 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:00:52.736412    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f\": container with ID starting with 77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f not found: ID does not exist" containerID="77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f"
Jul 24 18:00:52.736476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.736434    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f"} err="failed to get container status \"77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f\": rpc error: code = NotFound desc = could not find container \"77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f\": container with ID starting with 77ad4a36536fcb652f749a8f6d0cf84aed9db9d6bd7cd0d5a207a4ba7f19567f not found: ID does not exist"
Jul 24 18:00:52.743296 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.743275    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"]
Jul 24 18:00:52.745867 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.745847    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-5c7cfc-6bd84579f8-jrpm8"]
Jul 24 18:00:52.756748 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.756729    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"]
Jul 24 18:00:52.761342 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:52.761322    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5c7cfc-b48bf6944-bncgl"]
Jul 24 18:00:53.504739 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:53.504709    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="503b5993-5781-46b0-8ea5-c328eb0905be" path="/var/lib/kubelet/pods/503b5993-5781-46b0-8ea5-c328eb0905be/volumes"
Jul 24 18:00:53.505112 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:00:53.505039    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3253f25-a9d7-4364-9b92-21e17d22e9f3" path="/var/lib/kubelet/pods/e3253f25-a9d7-4364-9b92-21e17d22e9f3/volumes"
Jul 24 18:01:03.561431 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:03.561346    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"]
Jul 24 18:01:03.561900 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:03.561604    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy" containerID="cri-o://d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721" gracePeriod=30
Jul 24 18:01:06.954321 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:06.954282    2562 patch_prober.go:28] interesting pod/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 18:01:06.954677 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:06.954344    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 18:01:08.797286 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.797259    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 18:01:08.798330 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.798308    2562 generic.go:356] "Generic (PLEG): container finished" podID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerID="d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721" exitCode=0
Jul 24 18:01:08.798454 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.798382    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" event={"ID":"12ee3260-69e7-4c59-b4f4-60ea81eed431","Type":"ContainerDied","Data":"d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721"}
Jul 24 18:01:08.798454 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.798435    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs" event={"ID":"12ee3260-69e7-4c59-b4f4-60ea81eed431","Type":"ContainerDied","Data":"46a4e00821b7010c4a850f45c8fb070636242e5d3de285a865a4fcf304d6d4cb"}
Jul 24 18:01:08.798544 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.798460    2562 scope.go:117] "RemoveContainer" containerID="d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721"
Jul 24 18:01:08.806378 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.806361    2562 scope.go:117] "RemoveContainer" containerID="3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e"
Jul 24 18:01:08.813734 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.813720    2562 scope.go:117] "RemoveContainer" containerID="d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721"
Jul 24 18:01:08.813971 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:01:08.813953    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721\": container with ID starting with d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721 not found: ID does not exist" containerID="d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721"
Jul 24 18:01:08.814086 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.813981    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721"} err="failed to get container status \"d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721\": rpc error: code = NotFound desc = could not find container \"d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721\": container with ID starting with d57c1cadf2ec29a2232c52fd33bf8d78aeb8f4f21d25634d687e2a5edc930721 not found: ID does not exist"
Jul 24 18:01:08.814086 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.814004    2562 scope.go:117] "RemoveContainer" containerID="3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e"
Jul 24 18:01:08.814298 ip-10-0-140-5 kubenswrapper[2562]: E0724 18:01:08.814282    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e\": container with ID starting with 3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e not found: ID does not exist" containerID="3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e"
Jul 24 18:01:08.814348 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.814304    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e"} err="failed to get container status \"3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e\": rpc error: code = NotFound desc = could not find container \"3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e\": container with ID starting with 3ea04dc02498cfbe562aa111817f90b9420e893a9fe54fde6adc3aa5396fa32e not found: ID does not exist"
Jul 24 18:01:08.877776 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.877708    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-envoy\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.877776 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.877739    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-credential-socket\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.877776 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.877765    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qpzl\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-kube-api-access-8qpzl\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.878011 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.877817    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ee3260-69e7-4c59-b4f4-60ea81eed431-istiod-ca-cert\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.878011 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.877887    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-token\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.878011 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.877920    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-certs\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.878011 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.877962    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-podinfo\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.878011 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.877999    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-data\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.878270 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.878059    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-socket\") pod \"12ee3260-69e7-4c59-b4f4-60ea81eed431\" (UID: \"12ee3260-69e7-4c59-b4f4-60ea81eed431\") "
Jul 24 18:01:08.878270 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.878057    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:01:08.878391 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.878294    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-credential-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:08.878391 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.878321    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:01:08.878494 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.878416    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12ee3260-69e7-4c59-b4f4-60ea81eed431-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:01:08.878556 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.878518    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:01:08.879548 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.879507    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-data" (OuterVolumeSpecName: "istio-data") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:01:08.880307 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.880282    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-token" (OuterVolumeSpecName: "istio-token") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:01:08.880615 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.880585    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:01:08.880615 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.880586    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 18:01:08.880615 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.880610    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-kube-api-access-8qpzl" (OuterVolumeSpecName: "kube-api-access-8qpzl") pod "12ee3260-69e7-4c59-b4f4-60ea81eed431" (UID: "12ee3260-69e7-4c59-b4f4-60ea81eed431"). InnerVolumeSpecName "kube-api-access-8qpzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:01:08.979147 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.979120    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-token\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:08.979147 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.979141    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-certs\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:08.979147 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.979150    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-podinfo\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:08.979322 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.979157    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-data\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:08.979322 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.979166    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-workload-socket\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:08.979322 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.979174    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ee3260-69e7-4c59-b4f4-60ea81eed431-istio-envoy\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:08.979322 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.979181    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qpzl\" (UniqueName: \"kubernetes.io/projected/12ee3260-69e7-4c59-b4f4-60ea81eed431-kube-api-access-8qpzl\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:08.979322 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:08.979190    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ee3260-69e7-4c59-b4f4-60ea81eed431-istiod-ca-cert\") on node \"ip-10-0-140-5.ec2.internal\" DevicePath \"\""
Jul 24 18:01:09.803441 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:09.803359    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"
Jul 24 18:01:09.827274 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:09.827247    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"]
Jul 24 18:01:09.830711 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:09.830687    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-5c7cfc-openshift-default-74f888c8b8-xv6hs"]
Jul 24 18:01:09.858174 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:09.858150    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-9c58cc7f8-zkhg9_358bf28d-78cf-4495-8a8d-b3086513b70c/manager/0.log"
Jul 24 18:01:10.454280 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:10.454252    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd74455c-zt55x_5e0c6ff1-ad9b-47e1-947d-57762a4704d9/copy-manifests/0.log"
Jul 24 18:01:10.460885 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:10.460849    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd74455c-zt55x_5e0c6ff1-ad9b-47e1-947d-57762a4704d9/manager/0.log"
Jul 24 18:01:10.922380 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:10.922302    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748570-rtqt7_6762fda9-de50-4a3f-80b3-57b92c778208/cleanup/0.log"
Jul 24 18:01:11.047904 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:11.047879    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748585-ptbnn_b42cd707-38b8-4e45-9ad8-29fb1b351331/cleanup/0.log"
Jul 24 18:01:11.503989 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:11.503948    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" path="/var/lib/kubelet/pods/12ee3260-69e7-4c59-b4f4-60ea81eed431/volumes"
Jul 24 18:01:13.731834 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:13.731800    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77_1a9555d6-bef2-49cd-8a69-ee304a259828/istio-proxy/0.log"
Jul 24 18:01:13.850969 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:13.850937    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-mkdsh_50dc0ead-de9d-4bdc-a91c-772a61d6858b/discovery/0.log"
Jul 24 18:01:14.182158 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:14.182083    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j6kgc_54ab0abd-73ed-4fb7-9250-1345d1437057/istio-proxy/1.log"
Jul 24 18:01:14.296200 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:14.296170    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vzv9n_eccc9437-3a8e-461e-9404-1b7069db6edf/payload-pre-processing/0.log"
Jul 24 18:01:15.346557 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:15.346507    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9x9nb_f27589c9-42d0-4436-8c96-9c8d23ae389b/main/0.log"
Jul 24 18:01:24.029486 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:24.029427    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ptxkr_8d915ffa-99b0-4ff7-abc6-ca4c0df718bb/global-pull-secret-syncer/0.log"
Jul 24 18:01:24.193539 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:24.193510    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4ngr7_6769c8c9-a903-4966-9eab-351a6ec580fc/konnectivity-agent/0.log"
Jul 24 18:01:24.292807 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:24.292705    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-5.ec2.internal_9c39321f7468a5658b1cdb40ffa446cd/haproxy/0.log"
Jul 24 18:01:32.330411 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:32.330377    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44cwx_dad5a54f-a819-4aa0-b5a0-97f614e52a9a/node-exporter/0.log"
Jul 24 18:01:32.350103 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:32.350082    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44cwx_dad5a54f-a819-4aa0-b5a0-97f614e52a9a/kube-rbac-proxy/0.log"
Jul 24 18:01:32.368181 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:32.368161    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-44cwx_dad5a54f-a819-4aa0-b5a0-97f614e52a9a/init-textfile/0.log"
Jul 24 18:01:34.038049 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038007    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"]
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038331    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038344    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038353    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038361    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038371    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76520277-f9ec-40f8-b147-71dedc7f8516" containerName="maas-api"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038378    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="76520277-f9ec-40f8-b147-71dedc7f8516" containerName="maas-api"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038386    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038391    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038398    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038403    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038409    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="503b5993-5781-46b0-8ea5-c328eb0905be" containerName="payload-processing"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038413    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="503b5993-5781-46b0-8ea5-c328eb0905be" containerName="payload-processing"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038420    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a417d53-57f1-45a9-a3a4-5ab3f7aded7d" containerName="payload-processing"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038425    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a417d53-57f1-45a9-a3a4-5ab3f7aded7d" containerName="payload-processing"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038431    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce7fe025-938f-43de-be9e-dbce1431d810" containerName="payload-pre-processing"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038439    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce7fe025-938f-43de-be9e-dbce1431d810" containerName="payload-pre-processing"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038452    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dfa0922-d592-48aa-adff-e3a542c54e5b" containerName="maas-api"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038458    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dfa0922-d592-48aa-adff-e3a542c54e5b" containerName="maas-api"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038463    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0253d215-8bb3-49a2-8945-fbc2b330b0c8" containerName="maas-api"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038469    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0253d215-8bb3-49a2-8945-fbc2b330b0c8" containerName="maas-api"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038477    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2519ca4-8872-4038-bb16-f2da6f087073" containerName="payload-processing"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038482    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2519ca4-8872-4038-bb16-f2da6f087073" containerName="payload-processing"
Jul 24 18:01:34.038476 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038487    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038492    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038503    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d971647a-5331-4212-b720-2f5f90243930" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038507    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d971647a-5331-4212-b720-2f5f90243930" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038513    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3253f25-a9d7-4364-9b92-21e17d22e9f3" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038521    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3253f25-a9d7-4364-9b92-21e17d22e9f3" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038529    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186c8710-409a-44ac-a5ab-fd57bb540034" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038536    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="186c8710-409a-44ac-a5ab-fd57bb540034" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038583    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce7fe025-938f-43de-be9e-dbce1431d810" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038590    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038597    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c95dd0-732f-4362-a869-d2484c966208" containerName="istio-proxy"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038607    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="503b5993-5781-46b0-8ea5-c328eb0905be" containerName="payload-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038616    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae6b699d-2c14-40f9-bbdb-a6dd6804c9ff" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038621    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dfa0922-d592-48aa-adff-e3a542c54e5b" containerName="maas-api"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038626    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038629    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="186c8710-409a-44ac-a5ab-fd57bb540034" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038634    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0253d215-8bb3-49a2-8945-fbc2b330b0c8" containerName="maas-api"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038638    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3253f25-a9d7-4364-9b92-21e17d22e9f3" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038643    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2519ca4-8872-4038-bb16-f2da6f087073" containerName="payload-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038648    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d971647a-5331-4212-b720-2f5f90243930" containerName="payload-pre-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038653    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a417d53-57f1-45a9-a3a4-5ab3f7aded7d" containerName="payload-processing"
Jul 24 18:01:34.039201 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.038658    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="76520277-f9ec-40f8-b147-71dedc7f8516" containerName="maas-api"
Jul 24 18:01:34.041800 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.041784    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.044457 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.044438    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g7vgp\"/\"kube-root-ca.crt\""
Jul 24 18:01:34.045544 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.045527    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g7vgp\"/\"default-dockercfg-ml5bw\""
Jul 24 18:01:34.045600 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.045534    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g7vgp\"/\"openshift-service-ca.crt\""
Jul 24 18:01:34.050057 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.050034    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"]
Jul 24 18:01:34.182572 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.182538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-podres\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.182749 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.182584    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-proc\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.182749 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.182606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-lib-modules\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.182749 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.182679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npkd4\" (UniqueName: \"kubernetes.io/projected/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-kube-api-access-npkd4\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.182749 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.182721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-sys\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283540 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npkd4\" (UniqueName: \"kubernetes.io/projected/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-kube-api-access-npkd4\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283733 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-sys\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283733 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283625    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-sys\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283733 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-podres\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283733 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283685    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-proc\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283733 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-lib-modules\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283975 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-proc\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283975 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283806    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-lib-modules\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.283975 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.283818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-podres\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.295308 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.295243    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npkd4\" (UniqueName: \"kubernetes.io/projected/c21f3147-e7df-4b6c-a18b-6c3c1969ea95-kube-api-access-npkd4\") pod \"perf-node-gather-daemonset-dkbg8\" (UID: \"c21f3147-e7df-4b6c-a18b-6c3c1969ea95\") " pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.352210 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.352181    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.485718 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.485682    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"]
Jul 24 18:01:34.489510 ip-10-0-140-5 kubenswrapper[2562]: W0724 18:01:34.489480    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc21f3147_e7df_4b6c_a18b_6c3c1969ea95.slice/crio-9141c7f9060cd6b2e11e3d1c1e464044fe21aec37e7a1a536327eb04aba9c9eb WatchSource:0}: Error finding container 9141c7f9060cd6b2e11e3d1c1e464044fe21aec37e7a1a536327eb04aba9c9eb: Status 404 returned error can't find the container with id 9141c7f9060cd6b2e11e3d1c1e464044fe21aec37e7a1a536327eb04aba9c9eb
Jul 24 18:01:34.904327 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.904227    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8" event={"ID":"c21f3147-e7df-4b6c-a18b-6c3c1969ea95","Type":"ContainerStarted","Data":"9815805379fd62409142fc038a3e68e06c95b302bd52e3df7ce098cea30b1850"}
Jul 24 18:01:34.904327 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.904291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8" event={"ID":"c21f3147-e7df-4b6c-a18b-6c3c1969ea95","Type":"ContainerStarted","Data":"9141c7f9060cd6b2e11e3d1c1e464044fe21aec37e7a1a536327eb04aba9c9eb"}
Jul 24 18:01:34.904327 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.904314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:34.924189 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:34.924129    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8" podStartSLOduration=0.924109654 podStartE2EDuration="924.109654ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:01:34.92324852 +0000 UTC m=+3083.894380711" watchObservedRunningTime="2026-07-24 18:01:34.924109654 +0000 UTC m=+3083.895241836"
Jul 24 18:01:37.197759 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:37.197725    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8khf_dde92a18-877a-423d-9e1b-ed03e6638e03/dns/0.log"
Jul 24 18:01:37.219737 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:37.219709    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v8khf_dde92a18-877a-423d-9e1b-ed03e6638e03/kube-rbac-proxy/0.log"
Jul 24 18:01:37.243769 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:37.243740    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bmq8m_c8135909-83dd-4abb-9121-2c9b7dbd77ec/dns-node-resolver/0.log"
Jul 24 18:01:37.905882 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:37.905853    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-45lzv_8ab674fe-1d10-41b0-90ce-eb0ddfae3042/node-ca/0.log"
Jul 24 18:01:38.880198 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:38.880165    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77_1a9555d6-bef2-49cd-8a69-ee304a259828/istio-proxy/0.log"
Jul 24 18:01:39.017580 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:39.017548    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-mkdsh_50dc0ead-de9d-4bdc-a91c-772a61d6858b/discovery/0.log"
Jul 24 18:01:39.083553 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:39.083524    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j6kgc_54ab0abd-73ed-4fb7-9250-1345d1437057/istio-proxy/0.log"
Jul 24 18:01:39.111008 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:39.110981    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j6kgc_54ab0abd-73ed-4fb7-9250-1345d1437057/istio-proxy/1.log"
Jul 24 18:01:39.139278 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:39.139202    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vzv9n_eccc9437-3a8e-461e-9404-1b7069db6edf/payload-pre-processing/0.log"
Jul 24 18:01:40.071508 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.071478    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hts7p_11a91422-52c6-44f1-b193-7f2ed40394ca/serve-healthcheck-canary/0.log"
Jul 24 18:01:40.578093 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.578063    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6c59z_3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e/kube-rbac-proxy/0.log"
Jul 24 18:01:40.595324 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.595295    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6c59z_3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e/exporter/0.log"
Jul 24 18:01:40.612968 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.612946    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6c59z_3078dc6e-e8fa-4b3f-af1b-cdf73cfd0c4e/extractor/0.log"
Jul 24 18:01:40.918142 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.918065    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g7vgp/perf-node-gather-daemonset-dkbg8"
Jul 24 18:01:40.971714 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.971682    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h9bpj"]
Jul 24 18:01:40.972049 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.972036    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="12ee3260-69e7-4c59-b4f4-60ea81eed431" containerName="istio-proxy"
Jul 24 18:01:40.974941 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.974922    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h9bpj"
Jul 24 18:01:40.977722 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.977703    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 18:01:40.977882 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.977867    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 18:01:40.978061 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.978041    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6d57b\""
Jul 24 18:01:40.987455 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:40.987430    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h9bpj"]
Jul 24 18:01:41.140058 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:41.140030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptpfq\" (UniqueName: \"kubernetes.io/projected/cd0626a3-c3f2-4aca-b02b-6babe32d4b8b-kube-api-access-ptpfq\") pod \"kuadrant-operator-catalog-h9bpj\" (UID: \"cd0626a3-c3f2-4aca-b02b-6babe32d4b8b\") " pod="kuadrant-system/kuadrant-operator-catalog-h9bpj"
Jul 24 18:01:41.241125 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:41.241086    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptpfq\" (UniqueName: \"kubernetes.io/projected/cd0626a3-c3f2-4aca-b02b-6babe32d4b8b-kube-api-access-ptpfq\") pod \"kuadrant-operator-catalog-h9bpj\" (UID: \"cd0626a3-c3f2-4aca-b02b-6babe32d4b8b\") " pod="kuadrant-system/kuadrant-operator-catalog-h9bpj"
Jul 24 18:01:41.249854 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:41.249822    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptpfq\" (UniqueName: \"kubernetes.io/projected/cd0626a3-c3f2-4aca-b02b-6babe32d4b8b-kube-api-access-ptpfq\") pod \"kuadrant-operator-catalog-h9bpj\" (UID: \"cd0626a3-c3f2-4aca-b02b-6babe32d4b8b\") " pod="kuadrant-system/kuadrant-operator-catalog-h9bpj"
Jul 24 18:01:41.283962 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:41.283922    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h9bpj"
Jul 24 18:01:41.415717 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:41.415692    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h9bpj"]
Jul 24 18:01:41.417956 ip-10-0-140-5 kubenswrapper[2562]: W0724 18:01:41.417924    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd0626a3_c3f2_4aca_b02b_6babe32d4b8b.slice/crio-29dec81f2c2b773cc00aaf2297e3f80d543d80053905d0fd439e3ba1dc3080df WatchSource:0}: Error finding container 29dec81f2c2b773cc00aaf2297e3f80d543d80053905d0fd439e3ba1dc3080df: Status 404 returned error can't find the container with id 29dec81f2c2b773cc00aaf2297e3f80d543d80053905d0fd439e3ba1dc3080df
Jul 24 18:01:41.935047 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:41.934998    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h9bpj" event={"ID":"cd0626a3-c3f2-4aca-b02b-6babe32d4b8b","Type":"ContainerStarted","Data":"29dec81f2c2b773cc00aaf2297e3f80d543d80053905d0fd439e3ba1dc3080df"}
Jul 24 18:01:43.641317 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:43.641289    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-9c58cc7f8-zkhg9_358bf28d-78cf-4495-8a8d-b3086513b70c/manager/0.log"
Jul 24 18:01:43.946586 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:43.946554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h9bpj" event={"ID":"cd0626a3-c3f2-4aca-b02b-6babe32d4b8b","Type":"ContainerStarted","Data":"1bcee5de5459ce3fd64e434e5f1f589712c4e73646eec6212d147c8fe00ca561"}
Jul 24 18:01:43.966519 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:43.966469    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h9bpj" podStartSLOduration=1.807219924 podStartE2EDuration="3.966454896s" totalImagesPullingTime="2.159234972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:01:43.965781709 +0000 UTC m=+3092.936913890" watchObservedRunningTime="2026-07-24 18:01:43.966454896 +0000 UTC m=+3092.937587112"
Jul 24 18:01:44.016968 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:44.016944    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd74455c-zt55x_5e0c6ff1-ad9b-47e1-947d-57762a4704d9/manager/0.log"
Jul 24 18:01:44.043574 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:44.043550    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5bcd74455c-zt55x_5e0c6ff1-ad9b-47e1-947d-57762a4704d9/copy-manifests/0.log"
Jul 24 18:01:44.930243 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:44.930217    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77_1a9555d6-bef2-49cd-8a69-ee304a259828/istio-proxy/0.log"
Jul 24 18:01:45.070288 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:45.070262    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-mkdsh_50dc0ead-de9d-4bdc-a91c-772a61d6858b/discovery/0.log"
Jul 24 18:01:45.126498 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:45.126461    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j6kgc_54ab0abd-73ed-4fb7-9250-1345d1437057/istio-proxy/0.log"
Jul 24 18:01:45.155640 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:45.155607    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j6kgc_54ab0abd-73ed-4fb7-9250-1345d1437057/istio-proxy/1.log"
Jul 24 18:01:45.175083 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:45.175062    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vzv9n_eccc9437-3a8e-461e-9404-1b7069db6edf/payload-pre-processing/0.log"
Jul 24 18:01:45.283162 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:45.283135    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57c7c45669-zx7rw_c34cf630-e0de-4ae5-ba20-d9767dcb39c3/manager/0.log"
Jul 24 18:01:46.518253 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:46.518212    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kwr77_1a9555d6-bef2-49cd-8a69-ee304a259828/istio-proxy/0.log"
Jul 24 18:01:46.661146 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:46.661117    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-mkdsh_50dc0ead-de9d-4bdc-a91c-772a61d6858b/discovery/0.log"
Jul 24 18:01:46.708288 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:46.708260    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j6kgc_54ab0abd-73ed-4fb7-9250-1345d1437057/istio-proxy/0.log"
Jul 24 18:01:46.733401 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:46.733377    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-j6kgc_54ab0abd-73ed-4fb7-9250-1345d1437057/istio-proxy/1.log"
Jul 24 18:01:46.759256 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:46.759235    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vzv9n_eccc9437-3a8e-461e-9404-1b7069db6edf/payload-pre-processing/0.log"
Jul 24 18:01:50.197273 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:50.197243    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-4qbc7_7a2e3f2b-769a-4187-ba23-ce268bdb7d9b/migrator/0.log"
Jul 24 18:01:50.220820 ip-10-0-140-5 kubenswrapper[2562]: I0724 18:01:50.220796    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-4qbc7_7a2e3f2b-769a-4187-ba23-ce268bdb7d9b/graceful-termination/0.log"