Sep 09 12:46:28.760755 ip-10-0-132-160 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 12:46:28.760767 ip-10-0-132-160 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 12:46:28.760773 ip-10-0-132-160 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 12:46:28.760907 ip-10-0-132-160 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 12:46:38.796018 ip-10-0-132-160 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 12:46:38.796034 ip-10-0-132-160 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 5be184bcf36f4b898f2de642ab28552c --
Sep 09 12:48:33.885955 ip-10-0-132-160 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 12:48:34.266577 ip-10-0-132-160 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:48:34.266577 ip-10-0-132-160 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 12:48:34.266577 ip-10-0-132-160 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:48:34.266577 ip-10-0-132-160 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 12:48:34.266577 ip-10-0-132-160 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:48:34.267910 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.267810    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 12:48:34.272976 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272947    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:34.272976 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272972    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:34.272976 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272976    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:34.272976 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272980    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:34.272976 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272983    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272987    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272990    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272993    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.272996    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273000    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273005    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273009    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273011    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273014    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273017    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273020    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273022    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273025    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273028    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273030    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273033    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273036    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273038    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273041    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:34.273188 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273044    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273046    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273052    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273054    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273057    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273059    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273062    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273065    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273067    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273106    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273109    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273113    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273116    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273118    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273123    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273127    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273130    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273133    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273135    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:34.273679 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273139    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273144    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273147    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273149    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273152    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273155    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273157    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273161    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273164    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273167    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273170    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273181    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273184    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273187    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273190    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273192    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273195    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273198    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273201    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:34.274130 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273203    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273206    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273208    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273211    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273214    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273218    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273221    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273224    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273227    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273230    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273233    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273235    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273238    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273241    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273243    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273247    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273249    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273251    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273254    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273256    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:34.274593 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273259    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273262    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273265    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273267    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273270    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273273    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273276    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273279    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273281    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273284    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273287    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273289    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273292    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273294    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273297    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273300    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.273302    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274050    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274057    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274060    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:34.275126 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274063    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274066    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274068    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274071    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274074    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274076    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274079    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274081    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274085    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274087    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274090    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274093    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274096    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274098    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274102    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274104    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274107    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274111    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274114    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274116    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:34.275606 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274119    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274121    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274125    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274127    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274130    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274134    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274137    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274140    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274142    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274145    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274148    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274151    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274153    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274156    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274158    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274161    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274164    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274166    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274169    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:34.276112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274172    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274174    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274176    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274179    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274181    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274184    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274189    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274193    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274199    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274202    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274205    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274208    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274211    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274214    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274217    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274219    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274222    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274224    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274227    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:34.276573 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274230    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274232    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274235    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274239    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274242    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274246    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274249    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274252    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274254    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274257    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274259    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274262    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274264    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274267    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274269    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274271    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274274    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274276    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274279    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:34.277057 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274282    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274284    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274287    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274290    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274293    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274296    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274299    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274301    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274304    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274306    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274309    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274312    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274314    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274317    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274320    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274322    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274325    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274327    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274330    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274745    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:34.277525 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274751    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274754    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274756    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274759    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274762    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274765    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274767    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274770    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274773    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274775    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274778    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274781    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274783    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274786    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274788    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274792    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274795    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274797    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274800    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274803    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:34.278024 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274805    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274808    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274810    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274813    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274816    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274818    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274821    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274825    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274829    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274832    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274835    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274838    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274842    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274844    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274847    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274850    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274852    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274855    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274857    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:34.278493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274860    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274862    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274865    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274869    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274873    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274877    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274880    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274883    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274887    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274891    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274893    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274896    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274899    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274902    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274904    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274907    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274909    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274912    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274915    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:34.278962 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274918    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274920    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274923    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274925    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274928    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274930    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274932    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274935    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274938    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274940    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274943    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274945    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274948    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274950    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274953    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274957    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274959    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274962    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274964    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274967    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:34.279423 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274969    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274972    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274974    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274977    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274980    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274982    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274985    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274987    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274990    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274992    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274995    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.274997    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275000    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275003    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275005    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275008    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275010    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275013    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275016    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275018    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:34.279961 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275095    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275130    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275140    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275145    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275149    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275152    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275157    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275161    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275164    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275167    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275171    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275174    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275178    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275181    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275184    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275187    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275191    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275194    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275199    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275202    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275205    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275208    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275212    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275215    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 12:48:34.280475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275218    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275222    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275225    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275229    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275232    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275234    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275238    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275240    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275245    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275248    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275251    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275253    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275257    2577 flags.go:64] FLAG: --enable-server="true"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275260    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275264    2577 flags.go:64] FLAG: --event-burst="100"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275267    2577 flags.go:64] FLAG: --event-qps="50"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275270    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275273    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275276    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275280    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275282    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275286    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275289    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275292    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275295    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 12:48:34.281189 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275299    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275302    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275304    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275307    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275310    2577 flags.go:64] FLAG: --feature-gates=""
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275314    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275317    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275320    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275323    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275326    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275329    2577 flags.go:64] FLAG: --help="false"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275332    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275335    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275338    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275341    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275344    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275348    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275350    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275353    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275356    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275359    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275361    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275364    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275367    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 09 12:48:34.281842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275370    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275373    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275376    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275379    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275382    2577 flags.go:64] FLAG: --lock-file=""
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275388    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275391    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275394    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275399    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275403    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275406    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275409    2577 flags.go:64] FLAG: --logging-format="text"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275412    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275415    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275418    2577 flags.go:64] FLAG: --manifest-url=""
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275421    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275425    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275428    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275432    2577 flags.go:64] FLAG: --max-pods="110"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275435    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275438    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275440    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275443    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275446    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275449    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 12:48:34.282443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275452    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275460    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275463    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275466    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275469    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275472    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275476    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275479    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275482    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275488    2577 flags.go:64] FLAG: --port="10250"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275491    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275494    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-095a4502805cf06b1"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275497    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275501    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275504    2577 flags.go:64] FLAG: --register-node="true"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275507    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275510    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275514    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275517    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275519    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275523    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275526    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275530    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275533    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275535    2577 flags.go:64] FLAG: --runonce="false"
Sep 09 12:48:34.283084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275538    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275541    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275544    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275547    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275550    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275552    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275556    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275559    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275561    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275564    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275567    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275570    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275573    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275575    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275578    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275583    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275588    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275591    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275595    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275598    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275602    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275605    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275608    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275611    2577 flags.go:64] FLAG: --v="2"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275615    2577 flags.go:64] FLAG: --version="false"
Sep 09 12:48:34.283749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275620    2577 flags.go:64] FLAG: --vmodule=""
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275623    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.275627    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275768    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275774    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275779    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275784    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275788    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275792    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275794    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275797    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275799    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275802    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275805    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275807    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275810    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275812    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275815    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275818    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275820    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:34.284364 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275823    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275825    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275828    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275832    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275835    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275837    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275840    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275844    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275846    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275850    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275854    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275857    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275860    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275863    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275866    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275868    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275871    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275873    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275877    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275879    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:34.284874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275882    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275885    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275887    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275890    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275893    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275895    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275898    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275900    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275902    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275905    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275908    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275910    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275913    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275916    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275918    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275922    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275925    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275927    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275930    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275934    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:34.285358 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275937    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275939    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275942    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275945    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275947    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275950    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275953    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275956    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275958    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275961    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275964    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275966    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275968    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275971    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275974    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275976    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275980    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275983    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275986    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275989    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:34.285852 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275992    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275995    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.275997    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276000    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276003    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276007    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276011    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276014    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276017    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276019    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276022    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276025    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276028    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276030    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276033    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276036    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276038    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276041    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276044    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276047    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:34.286341 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276050    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.276052    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.276657    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.283898    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.283914    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.286024    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.286032    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286095    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286100    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286104    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286107    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286109    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286112    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286115    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:34.286842 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286118    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286121    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286124    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286127    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286129    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286132    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286134    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286137    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286139    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286142    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286144    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286147    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286149    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286152    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286154    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286157    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286160    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286167    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286171    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286173    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:34.287234 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286176    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286179    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286182    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286185    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286188    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286191    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286193    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286211    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286215    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286219    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286221    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286225    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286227    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286231    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286235    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286238    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286241    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286244    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286247    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286250    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:34.287749 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286252    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286255    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286257    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286260    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286262    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286264    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286267    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286269    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286272    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286275    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286278    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286281    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286284    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286287    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286292    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286295    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286297    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286300    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286302    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:34.288226 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286305    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286307    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286310    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286312    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286315    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286318    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286320    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286323    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286325    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286328    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286331    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286333    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286336    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286338    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286340    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286343    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286346    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286348    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286351    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:34.288775 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286355    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286359    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286363    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286366    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286369    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286372    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286375    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286378    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286382    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286385    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286388    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286390    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286393    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286396    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:34.289256 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.286402    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286514    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286518    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286521    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286523    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286526    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286529    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286532    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286535    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286538    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286541    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286544    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286547    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286549    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286552    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286554    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286557    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286559    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286563    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286565    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:34.289597 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286568    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286570    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286573    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286575    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286579    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286581    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286584    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286586    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286589    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286591    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286594    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286596    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286598    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286601    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286603    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286606    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286608    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286610    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286613    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286615    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:34.290098 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286618    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286620    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286623    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286625    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286627    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286630    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286649    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286652    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286656    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286659    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286663    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286666    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286669    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286671    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286675    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286679    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286682    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286685    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286688    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:34.290577 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286691    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286694    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286696    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286699    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286701    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286704    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286706    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286709    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286711    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286714    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286717    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286719    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286722    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286724    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286727    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286730    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286732    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286735    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286737    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:34.291047 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286739    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286742    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286744    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286747    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286749    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286752    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286754    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286758    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286762    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286764    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286768    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286771    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286774    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286777    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286779    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286782    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286784    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286786    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286789    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:34.291493 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286791    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:34.291969 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286794    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:34.291969 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:34.286796    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:34.291969 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.286801    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:48:34.291969 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.286909    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 12:48:34.291969 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.288927    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 12:48:34.291969 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.289934    2577 server.go:1019] "Starting client certificate rotation"
Sep 09 12:48:34.291969 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.290036    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:48:34.291969 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.290076    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:48:34.292310 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.292296    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:48:34.293593 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.293578    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:48:34.307812 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.307786    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 09 12:48:34.312078 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.312057    2577 log.go:25] "Validated CRI v1 image API"
Sep 09 12:48:34.313053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.313033    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 12:48:34.316928 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.316904    2577 fs.go:135] Filesystem UUIDs: map[2ccc44d7-f573-4a13-9dfb-40fae351f844:/dev/nvme0n1p4 7911ff05-2167-42b9-9a07-85c7973ccf99:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 09 12:48:34.317019 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.316927    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 12:48:34.321058 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.321035    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:48:34.323474 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.323347    2577 manager.go:216] Machine: {Timestamp:2026-09-09 12:48:34.321766557 +0000 UTC m=+0.322432648 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3085082 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec228821c3f76fa7b5f4452625c67015 SystemUUID:ec228821-c3f7-6fa7-b5f4-452625c67015 BootID:5be184bc-f36f-4b89-8f2d-e642ab28552c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:29:ea:4a:6c:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:29:ea:4a:6c:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:6f:5f:cb:ef:2f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 12:48:34.323474 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.323467    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 12:48:34.323625 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.323587    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 12:48:34.327053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.327028    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 12:48:34.327246 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.327057    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 12:48:34.327925 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.327913    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 12:48:34.327988 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.327932    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 12:48:34.327988 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.327943    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:48:34.329047 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.329034    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:48:34.330194 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.330182    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 12:48:34.333882 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.333869    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:48:34.334022 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.334010    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 12:48:34.334077 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.334052    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 12:48:34.334077 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.334074    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 12:48:34.334174 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.334096    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 12:48:34.334174 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.334113    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 09 12:48:34.334174 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.334127    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 12:48:34.334547 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.334532    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:48:34.334683 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.334558    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:48:34.336959 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.336943    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 12:48:34.338947 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.338934    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 12:48:34.338986 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.338964    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 12:48:34.339159 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339149    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 12:48:34.339197 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339164    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 12:48:34.339197 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339173    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 12:48:34.339197 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339181    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 12:48:34.339197 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339191    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 12:48:34.339197 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339196    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 12:48:34.339693 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339682    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 12:48:34.339729 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339696    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 12:48:34.339729 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339708    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 12:48:34.339729 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339716    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 12:48:34.339809 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339737    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 12:48:34.339809 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.339749    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 12:48:34.341025 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.341012    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 12:48:34.341066 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.341027    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 12:48:34.343904 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.343708    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pbqvf"
Sep 09 12:48:34.345485 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.345460    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 12:48:34.345586 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.345532    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 12:48:34.345659 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.345608    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 12:48:34.346369 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.346355    2577 server.go:1268] "Started kubelet"
Sep 09 12:48:34.346604 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.346569    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 12:48:34.346727 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.346675    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 12:48:34.346785 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.346750    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 12:48:34.347717 ip-10-0-132-160 systemd[1]: Started Kubernetes Kubelet.
Sep 09 12:48:34.347864 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.347720    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 12:48:34.348887 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.348872    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 12:48:34.350805 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.350777    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pbqvf"
Sep 09 12:48:34.351174 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.351160    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 12:48:34.351273 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.351254    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:34.351273 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.351164    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 12:48:34.351769 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.350717    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-160.ec2.internal.18d3a7ed3ba71be8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-160.ec2.internal,UID:ip-10-0-132-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-160.ec2.internal,},FirstTimestamp:2026-09-09 12:48:34.346114024 +0000 UTC m=+0.346780114,LastTimestamp:2026-09-09 12:48:34.346114024 +0000 UTC m=+0.346780114,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-160.ec2.internal,}"
Sep 09 12:48:34.351769 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.351744    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 12:48:34.351912 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.351784    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 12:48:34.351912 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.351799    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 12:48:34.351912 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.351870    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 12:48:34.351912 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.351891    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 12:48:34.352607 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.352580    2577 factory.go:55] Registering systemd factory
Sep 09 12:48:34.352607 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.352601    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 09 12:48:34.352798 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.352706    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 12:48:34.352894 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.352870    2577 factory.go:153] Registering CRI-O factory
Sep 09 12:48:34.352894 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.352891    2577 factory.go:223] Registration of the crio container factory successfully
Sep 09 12:48:34.352990 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.352933    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 12:48:34.352990 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.352947    2577 factory.go:103] Registering Raw factory
Sep 09 12:48:34.352990 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.352956    2577 manager.go:1196] Started watching for new ooms in manager
Sep 09 12:48:34.353342 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.353330    2577 manager.go:349] Starting recovery of all containers
Sep 09 12:48:34.360722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.360530    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:34.361352 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.361338    2577 manager.go:354] Recovery completed
Sep 09 12:48:34.366558 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.366537    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-160.ec2.internal\" not found" node="ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.368357 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.368345    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:34.370509 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.370490    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:34.370592 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.370522    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:34.370592 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.370533    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:34.371129 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.371114    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:34.371203 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.371114    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 12:48:34.371203 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.371148    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 12:48:34.371203 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.371168    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:48:34.371880 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.371861    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:34.371945 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.371898    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:34.371945 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.371914    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:34.373368 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.373355    2577 policy_none.go:49] "None policy: Start"
Sep 09 12:48:34.373428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.373374    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 12:48:34.373428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.373390    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 12:48:34.375487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.375473    2577 policy_none.go:47] "Start"
Sep 09 12:48:34.416355 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.416338    2577 manager.go:337] "Starting Device Plugin manager"
Sep 09 12:48:34.416455 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.416385    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 12:48:34.416455 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.416395    2577 server.go:87] "Starting device plugin registration server"
Sep 09 12:48:34.416762 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.416751    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 12:48:34.416823 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.416766    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 12:48:34.416876 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.416854    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 12:48:34.417009 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.416983    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 12:48:34.417009 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.416999    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 12:48:34.417454 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.417438    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 12:48:34.417525 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.417489    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:34.507740 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.507694    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 12:48:34.509842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.509087    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 12:48:34.509842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.509150    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 12:48:34.509842 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.509189    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 12:48:34.509842 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.509231    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 12:48:34.511699 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.511672    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:34.517605 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.517536    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:34.518376 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.518350    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:34.518467 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.518386    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:34.518467 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.518400    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:34.518467 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.518423    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.526218 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.526202    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.526281 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.526226    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-160.ec2.internal\": node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:34.535811 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.535794    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:34.609856 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.609812    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal"]
Sep 09 12:48:34.610015 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.609902    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:34.611502 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.611485    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:34.611584 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.611517    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:34.611584 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.611532    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:34.613818 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.613805    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:34.613951 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.613937    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.613991 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.613972    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:34.614599 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.614581    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:34.614715 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.614609    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:34.614715 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.614619    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:34.614715 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.614581    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:34.614715 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.614683    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:34.614715 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.614695    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:34.616754 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.616740    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.616838 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.616764    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:34.617407 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.617391    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:34.617489 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.617418    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:34.617489 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.617427    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:34.636075 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.636052    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:34.638314 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.638298    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-160.ec2.internal\" not found" node="ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.642573 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.642558    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-160.ec2.internal\" not found" node="ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.653372 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.653352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1caa5182608749a1cb38bd2b39488b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal\" (UID: \"a1caa5182608749a1cb38bd2b39488b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.653431 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.653376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1caa5182608749a1cb38bd2b39488b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal\" (UID: \"a1caa5182608749a1cb38bd2b39488b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.653431 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.653395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58beafae37c3742927ee4ffcc21b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-132-160.ec2.internal\" (UID: \"e58beafae37c3742927ee4ffcc21b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.736724 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.736694    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:34.754033 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.754004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1caa5182608749a1cb38bd2b39488b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal\" (UID: \"a1caa5182608749a1cb38bd2b39488b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.754105 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.754038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58beafae37c3742927ee4ffcc21b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-132-160.ec2.internal\" (UID: \"e58beafae37c3742927ee4ffcc21b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.754105 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.754056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1caa5182608749a1cb38bd2b39488b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal\" (UID: \"a1caa5182608749a1cb38bd2b39488b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.754105 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.754089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e58beafae37c3742927ee4ffcc21b4b4-config\") pod \"kube-apiserver-proxy-ip-10-0-132-160.ec2.internal\" (UID: \"e58beafae37c3742927ee4ffcc21b4b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.754105 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.754095    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1caa5182608749a1cb38bd2b39488b2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal\" (UID: \"a1caa5182608749a1cb38bd2b39488b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.754239 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.754123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a1caa5182608749a1cb38bd2b39488b2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal\" (UID: \"a1caa5182608749a1cb38bd2b39488b2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.837550 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.837477    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:34.938063 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:34.938020    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:34.941180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.941162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:34.944968 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:34.944952    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:35.038693 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.038656    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:35.139207 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.139136    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-160.ec2.internal\" not found"
Sep 09 12:48:35.208461 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.208436    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:35.251628 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.251600    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:35.265443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.265425    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:48:35.267259 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.267244    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal"
Sep 09 12:48:35.278989 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.278971    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:48:35.290117 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.290100    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 12:48:35.290240 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.290225    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:48:35.290306 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.290278    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:48:35.290349 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.290287    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:48:35.334440 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.334410    2577 apiserver.go:52] "Watching apiserver"
Sep 09 12:48:35.351489 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.351459    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 12:48:35.356458 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.356426    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 12:43:34 +0000 UTC" deadline="2028-03-12 00:17:12.509037091 +0000 UTC"
Sep 09 12:48:35.356458 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.356452    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13187h28m37.152587606s"
Sep 09 12:48:35.359016 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.358999    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 12:48:35.359965 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.359947    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-d8b9b","openshift-image-registry/node-ca-2cjcv","openshift-multus/multus-additional-cni-plugins-xwhz9","openshift-multus/multus-sg8rl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb","openshift-dns/node-resolver-dgkg9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal","openshift-multus/network-metrics-daemon-gjknn","openshift-network-diagnostics/network-check-target-qg547","openshift-network-operator/iptables-alerter-g52x6","openshift-ovn-kubernetes/ovnkube-node-djzps","kube-system/global-pull-secret-syncer-wvkhv","kube-system/konnectivity-agent-pnkgb","kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal"]
Sep 09 12:48:35.362716 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.362700    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.365605 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.365339    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 12:48:35.365605 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.365339    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4nvpl\""
Sep 09 12:48:35.365605 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.365508    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:35.366736 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.366717    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:48:35.368095 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.368078    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:35.369369 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.369353    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.371071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.371054    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 12:48:35.371255 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.371242    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ckzvl\""
Sep 09 12:48:35.372150 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.372137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.374728 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.374713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.375714 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.375691    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 12:48:35.375883 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.375868    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 12:48:35.375955 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.375918    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 12:48:35.376011 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.375979    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 12:48:35.376062 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.376031    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 12:48:35.376111 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.376100    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 12:48:35.376169 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.376141    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 12:48:35.376263 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.376248    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qjprr\""
Sep 09 12:48:35.376313 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.376255    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zjbzx\""
Sep 09 12:48:35.377409 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.377392    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 12:48:35.377489 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.377446    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.378109 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.378073    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 12:48:35.378178 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.378073    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t65pf\""
Sep 09 12:48:35.378178 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.378081    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 12:48:35.379497 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.379480    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:48:35.380048 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.380030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:35.380118 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.380097    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:35.380411 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.380398    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 12:48:35.380716 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.380704    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 12:48:35.382844 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.382827    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:35.382906 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.382885    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:35.383248 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.383232    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5wpd4\""
Sep 09 12:48:35.385382 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.385363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.385771 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.385758    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:35.388083 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.388069    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.390775 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.390731    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.391352 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.391331    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 12:48:35.391673 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.391660    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:35.391772 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.391757    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 12:48:35.391836 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.391800    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:35.391909 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.391895    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4pllp\""
Sep 09 12:48:35.391997 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.391982    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5v55n\""
Sep 09 12:48:35.392086 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.392074    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 12:48:35.393460 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.393444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.393539 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.393494    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:35.401053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.401035    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 12:48:35.401951 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.401936    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 12:48:35.402226 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.402211    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 12:48:35.402306 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.402212    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 12:48:35.402306 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.402275    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 12:48:35.402418 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.402338    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 12:48:35.402418 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.402344    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fm9qw\""
Sep 09 12:48:35.418530 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.418508    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s8t6p"
Sep 09 12:48:35.425209 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.425188    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s8t6p"
Sep 09 12:48:35.453688 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.453616    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 12:48:35.457828 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457807    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-system-cni-dir\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.457929 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-cni-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.457929 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-cnibin\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.457929 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-cni-bin\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.457929 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbad6c3c-19dc-4d5c-a127-7f47350d618a-host\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.457929 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457918    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hfw2\" (UniqueName: \"kubernetes.io/projected/8740aa13-1d21-498b-b8b3-00c783efa370-kube-api-access-7hfw2\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.458112 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-etc-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.458112 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bbad6c3c-19dc-4d5c-a127-7f47350d618a-serviceca\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.458112 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.457994    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2e18a6d-9313-4b12-8109-61a4d930b9e3-dbus\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.458112 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-host-slash\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.458112 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-netns\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.458112 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nccrv\" (UniqueName: \"kubernetes.io/projected/838a3ce3-e142-46db-9895-770fda4ef1da-kube-api-access-nccrv\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.458112 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-socket-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.458112 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-etc-selinux\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f42xv\" (UniqueName: \"kubernetes.io/projected/cf3fd93c-f29a-4477-88a7-620311730106-kube-api-access-f42xv\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5wsz\" (UniqueName: \"kubernetes.io/projected/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-kube-api-access-v5wsz\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-host\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa65dcab-8e63-4217-8118-0ae75f5dd140-konnectivity-ca\") pod \"konnectivity-agent-pnkgb\" (UID: \"fa65dcab-8e63-4217-8118-0ae75f5dd140\") " pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovnkube-config\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-cni-multus\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-iptables-alerter-script\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj7ln\" (UniqueName: \"kubernetes.io/projected/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-kube-api-access-lj7ln\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-var-lib-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/838a3ce3-e142-46db-9895-770fda4ef1da-cni-binary-copy\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-socket-dir-parent\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-hostroot\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.458384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-conf-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/838a3ce3-e142-46db-9895-770fda4ef1da-multus-daemon-config\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-registration-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-device-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458484    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmbn2\" (UniqueName: \"kubernetes.io/projected/270732e6-0ab3-42bc-a918-de8d94beeb1a-kube-api-access-rmbn2\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnx6t\" (UniqueName: \"kubernetes.io/projected/bbad6c3c-19dc-4d5c-a127-7f47350d618a-kube-api-access-fnx6t\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458515    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2e18a6d-9313-4b12-8109-61a4d930b9e3-kubelet-config\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458538    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-systemd\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458551    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-env-overrides\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8740aa13-1d21-498b-b8b3-00c783efa370-etc-tuned\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa65dcab-8e63-4217-8118-0ae75f5dd140-agent-certs\") pod \"konnectivity-agent-pnkgb\" (UID: \"fa65dcab-8e63-4217-8118-0ae75f5dd140\") " pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-hosts-file\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-run\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.458953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-sys\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-kubelet\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-systemd-units\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-node-log\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-cni-netd\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458905    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-cnibin\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-k8s-cni-cncf-io\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-sys-fs\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.458980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vncw\" (UniqueName: \"kubernetes.io/projected/8defff11-732a-4982-ae0c-d2f042889842-kube-api-access-4vncw\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-lib-modules\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459024    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-log-socket\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459051    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-tmp-dir\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-ovn\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-run-ovn-kubernetes\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.459496 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8740aa13-1d21-498b-b8b3-00c783efa370-tmp\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-slash\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-cni-bin\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-system-cni-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-kubelet\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-multus-certs\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-etc-kubernetes\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459277    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-var-lib-kubelet\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovnkube-script-lib\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysconfig\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-kubernetes\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysctl-d\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-run-netns\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.460071 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovn-node-metrics-cert\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.460706 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-modprobe-d\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.460706 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysctl-conf\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.460706 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-systemd\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.460706 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.459530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qkl6\" (UniqueName: \"kubernetes.io/projected/bd1a0a59-fc6b-434e-ba42-2e177c488d11-kube-api-access-7qkl6\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.560543 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-system-cni-dir\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.560670 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-cni-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.560670 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-cnibin\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.560670 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560630    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-cni-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.560670 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-system-cni-dir\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.560807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-cni-bin\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.560807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbad6c3c-19dc-4d5c-a127-7f47350d618a-host\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.560807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hfw2\" (UniqueName: \"kubernetes.io/projected/8740aa13-1d21-498b-b8b3-00c783efa370-kube-api-access-7hfw2\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.560807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560719    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-cnibin\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.560807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-etc-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.560807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-cni-bin\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.560807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bbad6c3c-19dc-4d5c-a127-7f47350d618a-serviceca\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.560807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560760    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbad6c3c-19dc-4d5c-a127-7f47350d618a-host\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-etc-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2e18a6d-9313-4b12-8109-61a4d930b9e3-dbus\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-host-slash\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560871    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2e18a6d-9313-4b12-8109-61a4d930b9e3-dbus\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-netns\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-host-slash\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-netns\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nccrv\" (UniqueName: \"kubernetes.io/projected/838a3ce3-e142-46db-9895-770fda4ef1da-kube-api-access-nccrv\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.560984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-socket-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-etc-selinux\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f42xv\" (UniqueName: \"kubernetes.io/projected/cf3fd93c-f29a-4477-88a7-620311730106-kube-api-access-f42xv\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:35.561053 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5wsz\" (UniqueName: \"kubernetes.io/projected/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-kube-api-access-v5wsz\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561099    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-etc-selinux\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561111    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bbad6c3c-19dc-4d5c-a127-7f47350d618a-serviceca\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-host\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-socket-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561154    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-host\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa65dcab-8e63-4217-8118-0ae75f5dd140-konnectivity-ca\") pod \"konnectivity-agent-pnkgb\" (UID: \"fa65dcab-8e63-4217-8118-0ae75f5dd140\") " pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.561487 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovnkube-config\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.561848 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561247    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.561898 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-cni-multus\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.561947 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-iptables-alerter-script\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.561993 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj7ln\" (UniqueName: \"kubernetes.io/projected/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-kube-api-access-lj7ln\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.561993 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.561979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-var-lib-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.562093 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/838a3ce3-e142-46db-9895-770fda4ef1da-cni-binary-copy\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.562093 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-socket-dir-parent\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.562093 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-hostroot\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.562093 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562083    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-var-lib-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.562275 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-conf-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.562275 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-conf-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.562275 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovnkube-config\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.562418 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-multus-socket-dir-parent\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.562418 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa65dcab-8e63-4217-8118-0ae75f5dd140-konnectivity-ca\") pod \"konnectivity-agent-pnkgb\" (UID: \"fa65dcab-8e63-4217-8118-0ae75f5dd140\") " pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:48:35.562418 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-hostroot\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.562418 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/838a3ce3-e142-46db-9895-770fda4ef1da-multus-daemon-config\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.562418 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-registration-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-device-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmbn2\" (UniqueName: \"kubernetes.io/projected/270732e6-0ab3-42bc-a918-de8d94beeb1a-kube-api-access-rmbn2\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnx6t\" (UniqueName: \"kubernetes.io/projected/bbad6c3c-19dc-4d5c-a127-7f47350d618a-kube-api-access-fnx6t\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2e18a6d-9313-4b12-8109-61a4d930b9e3-kubelet-config\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-systemd\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-env-overrides\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8740aa13-1d21-498b-b8b3-00c783efa370-etc-tuned\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa65dcab-8e63-4217-8118-0ae75f5dd140-agent-certs\") pod \"konnectivity-agent-pnkgb\" (UID: \"fa65dcab-8e63-4217-8118-0ae75f5dd140\") " pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:48:35.562722 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-hosts-file\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-iptables-alerter-script\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-run\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-sys\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/838a3ce3-e142-46db-9895-770fda4ef1da-multus-daemon-config\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562867    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-kubelet\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-openvswitch\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-systemd\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-kubelet\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-registration-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563132    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-device-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.562425    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-cni-multus\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.563180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563180    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-systemd-units\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563204    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2e18a6d-9313-4b12-8109-61a4d930b9e3-kubelet-config\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563212    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-node-log\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-hosts-file\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/838a3ce3-e142-46db-9895-770fda4ef1da-cni-binary-copy\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-run\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563368    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-systemd-units\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563309    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-env-overrides\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-cni-netd\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-cnibin\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-node-log\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-sys\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-k8s-cni-cncf-io\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-sys-fs\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-cni-netd\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/270732e6-0ab3-42bc-a918-de8d94beeb1a-cnibin\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.563884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vncw\" (UniqueName: \"kubernetes.io/projected/8defff11-732a-4982-ae0c-d2f042889842-kube-api-access-4vncw\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563610    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-k8s-cni-cncf-io\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563595    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-lib-modules\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-sys-fs\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-log-socket\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8defff11-732a-4982-ae0c-d2f042889842-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-log-socket\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-tmp-dir\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563810    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-lib-modules\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.563835    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-ovn\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-run-ovn-kubernetes\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8740aa13-1d21-498b-b8b3-00c783efa370-tmp\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.563918    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs podName:cf3fd93c-f29a-4477-88a7-620311730106 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:36.063896974 +0000 UTC m=+2.064563073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs") pod "network-metrics-daemon-gjknn" (UID: "cf3fd93c-f29a-4477-88a7-620311730106") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.563962    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-ovn\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.564802 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-slash\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-cni-bin\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-system-cni-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564129    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-run-ovn-kubernetes\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-kubelet\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-tmp-dir\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564175    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-multus-certs\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-var-lib-kubelet\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-etc-kubernetes\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564233    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-slash\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-var-lib-kubelet\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-cni-bin\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-var-lib-kubelet\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-system-cni-dir\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564444    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-host-run-multus-certs\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564452    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/838a3ce3-e142-46db-9895-770fda4ef1da-etc-kubernetes\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:35.565793 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovnkube-script-lib\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysconfig\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-kubernetes\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysctl-d\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564591    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysconfig\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.564531    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-kubernetes\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.564685    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret podName:d2e18a6d-9313-4b12-8109-61a4d930b9e3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:36.064667345 +0000 UTC m=+2.065333424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret") pod "global-pull-secret-syncer-wvkhv" (UID: "d2e18a6d-9313-4b12-8109-61a4d930b9e3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564750    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-run-netns\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564767    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/270732e6-0ab3-42bc-a918-de8d94beeb1a-cni-binary-copy\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovn-node-metrics-cert\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysctl-d\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-modprobe-d\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-host-run-netns\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysctl-conf\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-systemd\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.566743 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-modprobe-d\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.567518 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.564958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qkl6\" (UniqueName: \"kubernetes.io/projected/bd1a0a59-fc6b-434e-ba42-2e177c488d11-kube-api-access-7qkl6\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.567518 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.565047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovnkube-script-lib\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.567518 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.565058    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8740aa13-1d21-498b-b8b3-00c783efa370-etc-sysctl-conf\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.567518 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.565108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bd1a0a59-fc6b-434e-ba42-2e177c488d11-run-systemd\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.568823 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.568681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8740aa13-1d21-498b-b8b3-00c783efa370-tmp\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.569192 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.569159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa65dcab-8e63-4217-8118-0ae75f5dd140-agent-certs\") pod \"konnectivity-agent-pnkgb\" (UID: \"fa65dcab-8e63-4217-8118-0ae75f5dd140\") " pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:48:35.569287 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.569268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8740aa13-1d21-498b-b8b3-00c783efa370-etc-tuned\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.571498 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.570789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd1a0a59-fc6b-434e-ba42-2e177c488d11-ovn-node-metrics-cert\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.571498 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.571405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hfw2\" (UniqueName: \"kubernetes.io/projected/8740aa13-1d21-498b-b8b3-00c783efa370-kube-api-access-7hfw2\") pod \"tuned-d8b9b\" (UID: \"8740aa13-1d21-498b-b8b3-00c783efa370\") " pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.571682 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.571504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f42xv\" (UniqueName: \"kubernetes.io/projected/cf3fd93c-f29a-4477-88a7-620311730106-kube-api-access-f42xv\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:35.571682 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.571670    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj7ln\" (UniqueName: \"kubernetes.io/projected/482bfe86-6cb5-4c04-a941-e3e81ff24ad6-kube-api-access-lj7ln\") pod \"node-resolver-dgkg9\" (UID: \"482bfe86-6cb5-4c04-a941-e3e81ff24ad6\") " pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.572425 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.572406    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5wsz\" (UniqueName: \"kubernetes.io/projected/53b1b59c-83ee-49b2-bf36-16eac9cdcf2c-kube-api-access-v5wsz\") pod \"iptables-alerter-g52x6\" (UID: \"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c\") " pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.572929 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.572906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nccrv\" (UniqueName: \"kubernetes.io/projected/838a3ce3-e142-46db-9895-770fda4ef1da-kube-api-access-nccrv\") pod \"multus-sg8rl\" (UID: \"838a3ce3-e142-46db-9895-770fda4ef1da\") " pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.574312 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.574251    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnx6t\" (UniqueName: \"kubernetes.io/projected/bbad6c3c-19dc-4d5c-a127-7f47350d618a-kube-api-access-fnx6t\") pod \"node-ca-2cjcv\" (UID: \"bbad6c3c-19dc-4d5c-a127-7f47350d618a\") " pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.574396 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.574291    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qkl6\" (UniqueName: \"kubernetes.io/projected/bd1a0a59-fc6b-434e-ba42-2e177c488d11-kube-api-access-7qkl6\") pod \"ovnkube-node-djzps\" (UID: \"bd1a0a59-fc6b-434e-ba42-2e177c488d11\") " pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.574693 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.574669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vncw\" (UniqueName: \"kubernetes.io/projected/8defff11-732a-4982-ae0c-d2f042889842-kube-api-access-4vncw\") pod \"aws-ebs-csi-driver-node-kcskb\" (UID: \"8defff11-732a-4982-ae0c-d2f042889842\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.574812 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.574668    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmbn2\" (UniqueName: \"kubernetes.io/projected/270732e6-0ab3-42bc-a918-de8d94beeb1a-kube-api-access-rmbn2\") pod \"multus-additional-cni-plugins-xwhz9\" (UID: \"270732e6-0ab3-42bc-a918-de8d94beeb1a\") " pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.574812 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.574733    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1caa5182608749a1cb38bd2b39488b2.slice/crio-2211ea0ff750f9724f321a48f3d75c72f94fa05a396424394d4b2d5ab45a0220 WatchSource:0}: Error finding container 2211ea0ff750f9724f321a48f3d75c72f94fa05a396424394d4b2d5ab45a0220: Status 404 returned error can't find the container with id 2211ea0ff750f9724f321a48f3d75c72f94fa05a396424394d4b2d5ab45a0220
Sep 09 12:48:35.574928 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.574908    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58beafae37c3742927ee4ffcc21b4b4.slice/crio-66df610a63d5e9ef70c6c7fe49548a06a74485d7fc2e6546edc114a2764ed2d6 WatchSource:0}: Error finding container 66df610a63d5e9ef70c6c7fe49548a06a74485d7fc2e6546edc114a2764ed2d6: Status 404 returned error can't find the container with id 66df610a63d5e9ef70c6c7fe49548a06a74485d7fc2e6546edc114a2764ed2d6
Sep 09 12:48:35.578259 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.578237    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:35.578259 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.578259    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:35.578394 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.578268    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2pcs5 for pod openshift-network-diagnostics/network-check-target-qg547: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:35.578394 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:35.578311    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5 podName:da5b3772-17b5-4963-bcf5-3872b4c99cd0 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:36.0782973 +0000 UTC m=+2.078963378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2pcs5" (UniqueName: "kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5") pod "network-check-target-qg547" (UID: "da5b3772-17b5-4963-bcf5-3872b4c99cd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:35.581477 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.581464    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:48:35.697966 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.697871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g52x6"
Sep 09 12:48:35.708134 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.708109    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b1b59c_83ee_49b2_bf36_16eac9cdcf2c.slice/crio-1d08cd6e69ecb311ad36eadc0bbe4991dc4f0c2c6bda679f788cece109865469 WatchSource:0}: Error finding container 1d08cd6e69ecb311ad36eadc0bbe4991dc4f0c2c6bda679f788cece109865469: Status 404 returned error can't find the container with id 1d08cd6e69ecb311ad36eadc0bbe4991dc4f0c2c6bda679f788cece109865469
Sep 09 12:48:35.717586 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.717566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:48:35.722506 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.722480    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xwhz9"
Sep 09 12:48:35.726527 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.726496    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa65dcab_8e63_4217_8118_0ae75f5dd140.slice/crio-e6cce1d5942ed2835c079c7f22ab881537e19475990e34f6d22f6bef7dfaee21 WatchSource:0}: Error finding container e6cce1d5942ed2835c079c7f22ab881537e19475990e34f6d22f6bef7dfaee21: Status 404 returned error can't find the container with id e6cce1d5942ed2835c079c7f22ab881537e19475990e34f6d22f6bef7dfaee21
Sep 09 12:48:35.731352 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.731192    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod270732e6_0ab3_42bc_a918_de8d94beeb1a.slice/crio-72e4b3979e558dea014bd2e471454a4e3ad9c35c57d36ece8f79d0835e976729 WatchSource:0}: Error finding container 72e4b3979e558dea014bd2e471454a4e3ad9c35c57d36ece8f79d0835e976729: Status 404 returned error can't find the container with id 72e4b3979e558dea014bd2e471454a4e3ad9c35c57d36ece8f79d0835e976729
Sep 09 12:48:35.745703 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.745685    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sg8rl"
Sep 09 12:48:35.756538 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.756512    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838a3ce3_e142_46db_9895_770fda4ef1da.slice/crio-e17f80bd40231d0f1efc10031c7c389cbfddec9e8ece8f219f749bb68ec991d3 WatchSource:0}: Error finding container e17f80bd40231d0f1efc10031c7c389cbfddec9e8ece8f219f749bb68ec991d3: Status 404 returned error can't find the container with id e17f80bd40231d0f1efc10031c7c389cbfddec9e8ece8f219f749bb68ec991d3
Sep 09 12:48:35.762535 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.762513    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb"
Sep 09 12:48:35.767398 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.767378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dgkg9"
Sep 09 12:48:35.770969 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.770942    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8defff11_732a_4982_ae0c_d2f042889842.slice/crio-6970ded52aa4e5782260c7dd119f9a0fa3a3e15fa37c3415e1a8a792ce4c5846 WatchSource:0}: Error finding container 6970ded52aa4e5782260c7dd119f9a0fa3a3e15fa37c3415e1a8a792ce4c5846: Status 404 returned error can't find the container with id 6970ded52aa4e5782260c7dd119f9a0fa3a3e15fa37c3415e1a8a792ce4c5846
Sep 09 12:48:35.776502 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.776479    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod482bfe86_6cb5_4c04_a941_e3e81ff24ad6.slice/crio-115ff99f16220ccddd6971b5ce5a9c8a0e008cf74ed5c91c32db2b412c59b827 WatchSource:0}: Error finding container 115ff99f16220ccddd6971b5ce5a9c8a0e008cf74ed5c91c32db2b412c59b827: Status 404 returned error can't find the container with id 115ff99f16220ccddd6971b5ce5a9c8a0e008cf74ed5c91c32db2b412c59b827
Sep 09 12:48:35.802830 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.802808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2cjcv"
Sep 09 12:48:35.811874 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.811848    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbad6c3c_19dc_4d5c_a127_7f47350d618a.slice/crio-5fb569557bfdc6dae13f13ca635d6313ed33086fa97d16e056e718a75fd23d2e WatchSource:0}: Error finding container 5fb569557bfdc6dae13f13ca635d6313ed33086fa97d16e056e718a75fd23d2e: Status 404 returned error can't find the container with id 5fb569557bfdc6dae13f13ca635d6313ed33086fa97d16e056e718a75fd23d2e
Sep 09 12:48:35.825791 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.825763    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d8b9b"
Sep 09 12:48:35.830708 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:35.830688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:48:35.835461 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.835433    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8740aa13_1d21_498b_b8b3_00c783efa370.slice/crio-87664714cd2ccbfae8267afc7da8639300ff070a041c4b9bf5ae78d26fedf515 WatchSource:0}: Error finding container 87664714cd2ccbfae8267afc7da8639300ff070a041c4b9bf5ae78d26fedf515: Status 404 returned error can't find the container with id 87664714cd2ccbfae8267afc7da8639300ff070a041c4b9bf5ae78d26fedf515
Sep 09 12:48:35.839421 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:48:35.839398    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1a0a59_fc6b_434e_ba42_2e177c488d11.slice/crio-dd32efc41eb58d1a2be517e0fd7ba76de447bb2ad239447482a7e2612bfc9725 WatchSource:0}: Error finding container dd32efc41eb58d1a2be517e0fd7ba76de447bb2ad239447482a7e2612bfc9725: Status 404 returned error can't find the container with id dd32efc41eb58d1a2be517e0fd7ba76de447bb2ad239447482a7e2612bfc9725
Sep 09 12:48:36.067627 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.067531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:36.067627 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.067566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:36.067847 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.067701    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:36.067847 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.067701    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:36.067847 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.067749    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret podName:d2e18a6d-9313-4b12-8109-61a4d930b9e3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:37.067737219 +0000 UTC m=+3.068403296 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret") pod "global-pull-secret-syncer-wvkhv" (UID: "d2e18a6d-9313-4b12-8109-61a4d930b9e3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:36.067847 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.067762    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs podName:cf3fd93c-f29a-4477-88a7-620311730106 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:37.067756202 +0000 UTC m=+3.068422280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs") pod "network-metrics-daemon-gjknn" (UID: "cf3fd93c-f29a-4477-88a7-620311730106") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:36.167874 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.167847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:36.168044 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.167965    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:36.168044 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.167983    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:36.168044 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.167992    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2pcs5 for pod openshift-network-diagnostics/network-check-target-qg547: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:36.168135 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.168054    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5 podName:da5b3772-17b5-4963-bcf5-3872b4c99cd0 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:37.168025796 +0000 UTC m=+3.168691893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pcs5" (UniqueName: "kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5") pod "network-check-target-qg547" (UID: "da5b3772-17b5-4963-bcf5-3872b4c99cd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:36.289546 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.289516    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:36.428082 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.427973    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:43:35 +0000 UTC" deadline="2028-05-08 18:54:43.830049068 +0000 UTC"
Sep 09 12:48:36.428082 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.428029    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14574h6m7.402024581s"
Sep 09 12:48:36.513457 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.513422    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:36.513674 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.513555    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:36.513990 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.513968    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:36.514090 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.514071    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:36.514152 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.514140    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:36.514238 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:36.514221    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:36.520815 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.520763    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerStarted","Data":"72e4b3979e558dea014bd2e471454a4e3ad9c35c57d36ece8f79d0835e976729"}
Sep 09 12:48:36.524019 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.523991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pnkgb" event={"ID":"fa65dcab-8e63-4217-8118-0ae75f5dd140","Type":"ContainerStarted","Data":"e6cce1d5942ed2835c079c7f22ab881537e19475990e34f6d22f6bef7dfaee21"}
Sep 09 12:48:36.527280 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.527257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal" event={"ID":"e58beafae37c3742927ee4ffcc21b4b4","Type":"ContainerStarted","Data":"66df610a63d5e9ef70c6c7fe49548a06a74485d7fc2e6546edc114a2764ed2d6"}
Sep 09 12:48:36.529205 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.529182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"dd32efc41eb58d1a2be517e0fd7ba76de447bb2ad239447482a7e2612bfc9725"}
Sep 09 12:48:36.533394 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.533369    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g52x6" event={"ID":"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c","Type":"ContainerStarted","Data":"1d08cd6e69ecb311ad36eadc0bbe4991dc4f0c2c6bda679f788cece109865469"}
Sep 09 12:48:36.536659 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.536610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal" event={"ID":"a1caa5182608749a1cb38bd2b39488b2","Type":"ContainerStarted","Data":"2211ea0ff750f9724f321a48f3d75c72f94fa05a396424394d4b2d5ab45a0220"}
Sep 09 12:48:36.540032 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.540007    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d8b9b" event={"ID":"8740aa13-1d21-498b-b8b3-00c783efa370","Type":"ContainerStarted","Data":"87664714cd2ccbfae8267afc7da8639300ff070a041c4b9bf5ae78d26fedf515"}
Sep 09 12:48:36.542870 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.542846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2cjcv" event={"ID":"bbad6c3c-19dc-4d5c-a127-7f47350d618a","Type":"ContainerStarted","Data":"5fb569557bfdc6dae13f13ca635d6313ed33086fa97d16e056e718a75fd23d2e"}
Sep 09 12:48:36.548845 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.548822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dgkg9" event={"ID":"482bfe86-6cb5-4c04-a941-e3e81ff24ad6","Type":"ContainerStarted","Data":"115ff99f16220ccddd6971b5ce5a9c8a0e008cf74ed5c91c32db2b412c59b827"}
Sep 09 12:48:36.554895 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.554868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb" event={"ID":"8defff11-732a-4982-ae0c-d2f042889842","Type":"ContainerStarted","Data":"6970ded52aa4e5782260c7dd119f9a0fa3a3e15fa37c3415e1a8a792ce4c5846"}
Sep 09 12:48:36.556882 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.556863    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sg8rl" event={"ID":"838a3ce3-e142-46db-9895-770fda4ef1da","Type":"ContainerStarted","Data":"e17f80bd40231d0f1efc10031c7c389cbfddec9e8ece8f219f749bb68ec991d3"}
Sep 09 12:48:36.647256 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:36.647217    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:37.066560 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:37.066527    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:37.077747 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:37.077715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:37.077908 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:37.077769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:37.078362 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:37.078150    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:37.078465 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:37.078434    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret podName:d2e18a6d-9313-4b12-8109-61a4d930b9e3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:39.078401115 +0000 UTC m=+5.079067267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret") pod "global-pull-secret-syncer-wvkhv" (UID: "d2e18a6d-9313-4b12-8109-61a4d930b9e3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:37.081834 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:37.081810    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:37.081935 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:37.081895    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs podName:cf3fd93c-f29a-4477-88a7-620311730106 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:39.08187443 +0000 UTC m=+5.082540514 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs") pod "network-metrics-daemon-gjknn" (UID: "cf3fd93c-f29a-4477-88a7-620311730106") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:37.178505 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:37.178462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:37.178726 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:37.178694    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:37.178726 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:37.178719    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:37.178852 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:37.178733    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2pcs5 for pod openshift-network-diagnostics/network-check-target-qg547: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:37.178852 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:37.178805    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5 podName:da5b3772-17b5-4963-bcf5-3872b4c99cd0 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:39.178787119 +0000 UTC m=+5.179453200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pcs5" (UniqueName: "kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5") pod "network-check-target-qg547" (UID: "da5b3772-17b5-4963-bcf5-3872b4c99cd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:37.428924 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:37.428853    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:43:35 +0000 UTC" deadline="2028-03-02 09:36:33.293737093 +0000 UTC"
Sep 09 12:48:37.428924 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:37.428921    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12956h47m55.864820214s"
Sep 09 12:48:38.512480 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:38.512403    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:38.512953 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:38.512578    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:38.513028 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:38.513011    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:38.513124 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:38.513106    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:38.513188 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:38.513172    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:38.513277 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:38.513260    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:39.095755 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:39.095713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:39.095939 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:39.095776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:39.096033 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:39.095941    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:39.096033 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:39.096010    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret podName:d2e18a6d-9313-4b12-8109-61a4d930b9e3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:43.095987899 +0000 UTC m=+9.096653980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret") pod "global-pull-secret-syncer-wvkhv" (UID: "d2e18a6d-9313-4b12-8109-61a4d930b9e3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:39.096432 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:39.096413    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:39.096521 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:39.096466    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs podName:cf3fd93c-f29a-4477-88a7-620311730106 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:43.096451199 +0000 UTC m=+9.097117282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs") pod "network-metrics-daemon-gjknn" (UID: "cf3fd93c-f29a-4477-88a7-620311730106") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:39.197200 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:39.197157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:39.197500 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:39.197346    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:39.197500 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:39.197368    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:39.197500 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:39.197381    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2pcs5 for pod openshift-network-diagnostics/network-check-target-qg547: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:39.197500 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:39.197453    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5 podName:da5b3772-17b5-4963-bcf5-3872b4c99cd0 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:43.197432936 +0000 UTC m=+9.198099017 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pcs5" (UniqueName: "kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5") pod "network-check-target-qg547" (UID: "da5b3772-17b5-4963-bcf5-3872b4c99cd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:40.511082 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:40.511047    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:40.511523 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:40.511170    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:40.511523 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:40.511309    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:40.511523 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:40.511422    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:40.511686 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:40.511561    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:40.511686 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:40.511663    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:42.511724 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:42.511686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:42.512207 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:42.511838    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:42.512289 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:42.512264    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:42.512380 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:42.512361    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:42.512445 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:42.512431    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:42.512522 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:42.512505    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:43.137222 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:43.137181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:43.137372 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:43.137238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:43.137450 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:43.137418    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:43.137553 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:43.137507    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs podName:cf3fd93c-f29a-4477-88a7-620311730106 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:51.137486148 +0000 UTC m=+17.138152236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs") pod "network-metrics-daemon-gjknn" (UID: "cf3fd93c-f29a-4477-88a7-620311730106") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:43.137553 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:43.137418    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:43.137664 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:43.137586    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret podName:d2e18a6d-9313-4b12-8109-61a4d930b9e3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:51.137567794 +0000 UTC m=+17.138233887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret") pod "global-pull-secret-syncer-wvkhv" (UID: "d2e18a6d-9313-4b12-8109-61a4d930b9e3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:43.238384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:43.238344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:43.238579 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:43.238548    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:43.238579 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:43.238584    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:43.238770 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:43.238599    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2pcs5 for pod openshift-network-diagnostics/network-check-target-qg547: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:43.238770 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:43.238685    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5 podName:da5b3772-17b5-4963-bcf5-3872b4c99cd0 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:51.238665148 +0000 UTC m=+17.239331243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pcs5" (UniqueName: "kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5") pod "network-check-target-qg547" (UID: "da5b3772-17b5-4963-bcf5-3872b4c99cd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:44.511360 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:44.511322    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:44.511940 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:44.511335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:44.511940 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:44.511475    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:44.511940 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:44.511344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:44.511940 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:44.511544    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:44.511940 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:44.511614    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:46.510419 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:46.510376    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:46.510419 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:46.510407    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:46.511049 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:46.510386    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:46.511049 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:46.510505    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:46.511049 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:46.510616    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:46.511049 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:46.510719    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:48.509925 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:48.509890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:48.510372 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:48.509890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:48.510372 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:48.510014    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:48.510372 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:48.510039    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:48.510372 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:48.510112    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:48.510372 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:48.510205    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:50.509866 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:50.509824    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:50.510384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:50.509824    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:50.510384 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:50.509974    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:50.510384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:50.509823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:50.510384 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:50.510049    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:50.510384 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:50.510104    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:51.204510 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:51.204480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:51.204700 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:51.204521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:51.204700 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:51.204661    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:51.204700 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:51.204661    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:51.204844 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:51.204729    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret podName:d2e18a6d-9313-4b12-8109-61a4d930b9e3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:07.20470972 +0000 UTC m=+33.205375812 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret") pod "global-pull-secret-syncer-wvkhv" (UID: "d2e18a6d-9313-4b12-8109-61a4d930b9e3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:51.204844 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:51.204748    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs podName:cf3fd93c-f29a-4477-88a7-620311730106 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:07.204738573 +0000 UTC m=+33.205404651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs") pod "network-metrics-daemon-gjknn" (UID: "cf3fd93c-f29a-4477-88a7-620311730106") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:51.305264 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:51.305228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:51.305534 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:51.305431    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:51.305534 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:51.305460    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:51.305534 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:51.305475    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2pcs5 for pod openshift-network-diagnostics/network-check-target-qg547: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:51.305691 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:51.305574    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5 podName:da5b3772-17b5-4963-bcf5-3872b4c99cd0 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:07.305525004 +0000 UTC m=+33.306191102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pcs5" (UniqueName: "kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5") pod "network-check-target-qg547" (UID: "da5b3772-17b5-4963-bcf5-3872b4c99cd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:52.509567 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:52.509534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:52.510063 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:52.509534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:52.510063 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:52.509748    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:52.510063 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:52.509534    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:52.510063 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:52.509661    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:52.510063 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:52.509889    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:54.510204 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:54.510169    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:54.510712 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:54.510279    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:54.510712 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:54.510604    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:54.510847 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:54.510741    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:54.510847 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:54.510803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:54.510954 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:54.510859    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:55.633898 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.633671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"1f503b7cba008be1614d89bc4f86e144b6e44dd11ee1cab154965a408532c059"}
Sep 09 12:48:55.634529 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.633977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"d1fda440815050042b352bbdac80902dfcdb4cbaccddd27c3e6222c04b5c7f45"}
Sep 09 12:48:55.634529 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.634014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"54fe54277378a61890fdd175a18a2568e274f9a5ef45011af61871f630930bae"}
Sep 09 12:48:55.636236 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.636202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d8b9b" event={"ID":"8740aa13-1d21-498b-b8b3-00c783efa370","Type":"ContainerStarted","Data":"fdecc274d1d53a003c2e601adb267e80823bc4dfb6be63390797bf6d96b848f5"}
Sep 09 12:48:55.638233 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.637852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sg8rl" event={"ID":"838a3ce3-e142-46db-9895-770fda4ef1da","Type":"ContainerStarted","Data":"6308c4758622ad22a44760be54f3dbd1f72379db90e27ac9041445a5e5ec8ad8"}
Sep 09 12:48:55.643066 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.643044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal" event={"ID":"e58beafae37c3742927ee4ffcc21b4b4","Type":"ContainerStarted","Data":"130d5df6b6eba8acbbf317e8f2bdb2a055176db96125b14a7bc0d3e29928fb80"}
Sep 09 12:48:55.652574 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.652519    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d8b9b" podStartSLOduration=2.244284022 podStartE2EDuration="21.652499554s" totalImagesPullingTime="19.408215532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:55.651880448 +0000 UTC m=+21.652546548" watchObservedRunningTime="2026-09-09 12:48:55.652499554 +0000 UTC m=+21.653165656"
Sep 09 12:48:55.668264 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.668213    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sg8rl" podStartSLOduration=2.105902997 podStartE2EDuration="21.66819317s" totalImagesPullingTime="19.562290173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:55.66756262 +0000 UTC m=+21.668228720" watchObservedRunningTime="2026-09-09 12:48:55.66819317 +0000 UTC m=+21.668859271"
Sep 09 12:48:55.680898 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:55.680843    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-160.ec2.internal" podStartSLOduration=20.680828404 podStartE2EDuration="20.680828404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:55.680248325 +0000 UTC m=+21.680914443" watchObservedRunningTime="2026-09-09 12:48:55.680828404 +0000 UTC m=+21.681494504"
Sep 09 12:48:56.510553 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.510325    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:56.510742 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.510332    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:56.510742 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:56.510576    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:56.510742 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:56.510687    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:56.510742 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.510332    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:56.510917 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:56.510792    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:56.647294 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.647254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb" event={"ID":"8defff11-732a-4982-ae0c-d2f042889842","Type":"ContainerStarted","Data":"4b17e6f8415b504079dadc59e2231765c1ba128c015144015175b9be23f5cbf9"}
Sep 09 12:48:56.648863 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.648839    2577 generic.go:356] "Generic (PLEG): container finished" podID="270732e6-0ab3-42bc-a918-de8d94beeb1a" containerID="2695fe8e3873248f33bbe63c673a4c01b0bb2b714c7cff56d7a0c74335b9094a" exitCode=0
Sep 09 12:48:56.648934 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.648894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerDied","Data":"2695fe8e3873248f33bbe63c673a4c01b0bb2b714c7cff56d7a0c74335b9094a"}
Sep 09 12:48:56.650482 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.650413    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pnkgb" event={"ID":"fa65dcab-8e63-4217-8118-0ae75f5dd140","Type":"ContainerStarted","Data":"ec838c77d2e6c90e74ef503349ce3e977f9634da2f6b56cb38cb2b738bb54912"}
Sep 09 12:48:56.654076 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.654056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"a7083464ba1a7fc99748eb54f063eaeacc0b4843161f13bd50b222a4f745c1bb"}
Sep 09 12:48:56.654234 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.654125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"94a713b5d490265c109687563d1f2977a132737e12af806e79842f93d1010ed5"}
Sep 09 12:48:56.654234 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.654163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"b878404ec4601374bf7f5e8394b57e1debee36732f2077228fe47439364f15d6"}
Sep 09 12:48:56.655548 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.655524    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1caa5182608749a1cb38bd2b39488b2" containerID="17e880b2723e857a20e4f1ac1fa95eea1b0780e9a0d8528250b8adae6e3860a3" exitCode=0
Sep 09 12:48:56.655678 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.655598    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal" event={"ID":"a1caa5182608749a1cb38bd2b39488b2","Type":"ContainerDied","Data":"17e880b2723e857a20e4f1ac1fa95eea1b0780e9a0d8528250b8adae6e3860a3"}
Sep 09 12:48:56.657032 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.657011    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2cjcv" event={"ID":"bbad6c3c-19dc-4d5c-a127-7f47350d618a","Type":"ContainerStarted","Data":"c245c74f14b0b0800a2182117029ad6f58032db9aff24c6d0070340393d5764f"}
Sep 09 12:48:56.658684 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.658394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dgkg9" event={"ID":"482bfe86-6cb5-4c04-a941-e3e81ff24ad6","Type":"ContainerStarted","Data":"93b4b96261de8cff8ded96f01f1312209ea6efe276fc66a37d50dbf8dde8bbd8"}
Sep 09 12:48:56.681431 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.681371    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pnkgb" podStartSLOduration=3.191818755 podStartE2EDuration="22.681351186s" totalImagesPullingTime="19.489532431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:56.680750536 +0000 UTC m=+22.681416637" watchObservedRunningTime="2026-09-09 12:48:56.681351186 +0000 UTC m=+22.682017287"
Sep 09 12:48:56.696154 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.696101    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2cjcv" podStartSLOduration=3.266766076 podStartE2EDuration="22.696081526s" totalImagesPullingTime="19.42931545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:56.695829696 +0000 UTC m=+22.696495797" watchObservedRunningTime="2026-09-09 12:48:56.696081526 +0000 UTC m=+22.696747626"
Sep 09 12:48:56.728165 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:56.728115    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dgkg9" podStartSLOduration=6.949045687 podStartE2EDuration="22.728098968s" totalImagesPullingTime="15.779053281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:56.727116547 +0000 UTC m=+22.727782648" watchObservedRunningTime="2026-09-09 12:48:56.728098968 +0000 UTC m=+22.728765068"
Sep 09 12:48:57.492000 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:57.491969    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 12:48:57.663407 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:57.663315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g52x6" event={"ID":"53b1b59c-83ee-49b2-bf36-16eac9cdcf2c","Type":"ContainerStarted","Data":"cd3eb0d262dd79300766c3db2e62afdc8fe5ceace23f5522b4dcbb03ceede963"}
Sep 09 12:48:57.665762 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:57.665736    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal" event={"ID":"a1caa5182608749a1cb38bd2b39488b2","Type":"ContainerStarted","Data":"2696cab9dc3b01239936060be52d46f5ffeaebfe3a8f315a590cbc9f764ed20c"}
Sep 09 12:48:57.667945 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:57.667916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb" event={"ID":"8defff11-732a-4982-ae0c-d2f042889842","Type":"ContainerStarted","Data":"e450a28840f31b199c5724433601803feb41ef0d167ea6084921dbe17b8f9406"}
Sep 09 12:48:57.677223 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:57.677169    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g52x6" podStartSLOduration=4.19020782 podStartE2EDuration="23.677152583s" totalImagesPullingTime="19.486944763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:57.677095285 +0000 UTC m=+23.677761386" watchObservedRunningTime="2026-09-09 12:48:57.677152583 +0000 UTC m=+23.677818685"
Sep 09 12:48:57.690266 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:57.690219    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-160.ec2.internal" podStartSLOduration=22.690201623 podStartE2EDuration="22.690201623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:57.690165988 +0000 UTC m=+23.690832089" watchObservedRunningTime="2026-09-09 12:48:57.690201623 +0000 UTC m=+23.690867724"
Sep 09 12:48:58.430653 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:58.430233    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T12:48:57.49199544Z","UUID":"b5286ef3-cbeb-4fcd-8805-37919b907038","Handler":null,"Name":"","Endpoint":""}
Sep 09 12:48:58.432428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:58.432401    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 12:48:58.432571 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:58.432441    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 12:48:58.510150 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:58.510116    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:48:58.510347 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:58.510116    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:48:58.510347 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:58.510234    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:48:58.510347 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:58.510238    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:48:58.510347 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:58.510322    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:48:58.510581 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:48:58.510391    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:48:58.673450 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:58.673413    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"f1070eadfef20f53a6dc7a59977d69a9e19d5d11c70d9892f8abceff5ebf6a37"}
Sep 09 12:48:59.678369 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:59.678327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb" event={"ID":"8defff11-732a-4982-ae0c-d2f042889842","Type":"ContainerStarted","Data":"d6bbaa4bac1e45869bfef028436ebf4858b3823e239b8e924a9535a493d9e9c9"}
Sep 09 12:48:59.695119 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:48:59.695073    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kcskb" podStartSLOduration=7.504925308 podStartE2EDuration="25.695058432s" totalImagesPullingTime="18.190133124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:59.694628479 +0000 UTC m=+25.695294578" watchObservedRunningTime="2026-09-09 12:48:59.695058432 +0000 UTC m=+25.695724532"
Sep 09 12:49:00.300000 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:00.299870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:49:00.301438 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:00.301417    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:49:00.509993 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:00.509910    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:00.510156 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:00.510058    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:49:00.510156 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:00.510132    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:00.510259 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:00.510223    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:49:00.510312 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:00.510263    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:00.510366 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:00.510327    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:49:00.685410 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:00.685366    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" event={"ID":"bd1a0a59-fc6b-434e-ba42-2e177c488d11","Type":"ContainerStarted","Data":"2558a22186292240b77a300cc5619f81d6d60dafde46aa529cf1d8f40cdda747"}
Sep 09 12:49:00.709864 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:00.709800    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-djzps" podStartSLOduration=7.274538004 podStartE2EDuration="26.709778827s" totalImagesPullingTime="19.435240823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:00.708480285 +0000 UTC m=+26.709146387" watchObservedRunningTime="2026-09-09 12:49:00.709778827 +0000 UTC m=+26.710444927"
Sep 09 12:49:01.688121 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:01.687893    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 12:49:01.688121 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:01.688074    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:49:01.688663 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:01.688628    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:49:01.707245 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:01.707212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:49:01.707414 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:01.707400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:49:02.196866 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.196834    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gjknn"]
Sep 09 12:49:02.197073 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.196996    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:02.197162 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:02.197122    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:49:02.199582 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.199547    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qg547"]
Sep 09 12:49:02.199726 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.199666    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:02.199782 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:02.199755    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:49:02.200197 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.200171    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wvkhv"]
Sep 09 12:49:02.200327 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.200271    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:02.200391 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:02.200371    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:49:02.692388 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.692288    2577 generic.go:356] "Generic (PLEG): container finished" podID="270732e6-0ab3-42bc-a918-de8d94beeb1a" containerID="337f1120ba714d1295aefe70befdc5d7ab08b4170f6fa02af0dd6859761f86ed" exitCode=0
Sep 09 12:49:02.693013 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.692377    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerDied","Data":"337f1120ba714d1295aefe70befdc5d7ab08b4170f6fa02af0dd6859761f86ed"}
Sep 09 12:49:02.693013 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:02.692540    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 12:49:03.509881 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:03.509849    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:03.510040 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:03.509953    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:49:03.697549 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:03.697511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerStarted","Data":"54a0d1b9150178adcf90e61d2fb49d23d92c7b5d94bec1a937d2e5f9b8b82739"}
Sep 09 12:49:03.698077 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:03.697572    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 12:49:03.999980 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:03.999885    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:49:04.000127 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:04.000021    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 12:49:04.000564 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:04.000548    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pnkgb"
Sep 09 12:49:04.510244 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:04.510207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:04.510450 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:04.510316    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:49:04.510450 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:04.510408    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:04.510569 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:04.510538    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:49:04.702161 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:04.701910    2577 generic.go:356] "Generic (PLEG): container finished" podID="270732e6-0ab3-42bc-a918-de8d94beeb1a" containerID="54a0d1b9150178adcf90e61d2fb49d23d92c7b5d94bec1a937d2e5f9b8b82739" exitCode=0
Sep 09 12:49:04.702538 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:04.701988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerDied","Data":"54a0d1b9150178adcf90e61d2fb49d23d92c7b5d94bec1a937d2e5f9b8b82739"}
Sep 09 12:49:04.865243 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:04.865206    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:49:05.509524 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:05.509490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:05.509721 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:05.509593    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:49:05.706340 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:05.706241    2577 generic.go:356] "Generic (PLEG): container finished" podID="270732e6-0ab3-42bc-a918-de8d94beeb1a" containerID="484878083b03abe98bbce83ca06e98fc8ba4f45f77c19f63329106dff95fc90a" exitCode=0
Sep 09 12:49:05.706725 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:05.706331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerDied","Data":"484878083b03abe98bbce83ca06e98fc8ba4f45f77c19f63329106dff95fc90a"}
Sep 09 12:49:06.510372 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:06.510333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:06.510572 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:06.510379    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:06.510572 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:06.510473    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gjknn" podUID="cf3fd93c-f29a-4477-88a7-620311730106"
Sep 09 12:49:06.510708 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:06.510606    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qg547" podUID="da5b3772-17b5-4963-bcf5-3872b4c99cd0"
Sep 09 12:49:07.225043 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:07.225005    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:07.225504 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:07.225107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:07.225504 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.225188    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:49:07.225504 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.225231    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:49:07.225504 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.225271    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret podName:d2e18a6d-9313-4b12-8109-61a4d930b9e3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:39.225251136 +0000 UTC m=+65.225917218 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret") pod "global-pull-secret-syncer-wvkhv" (UID: "d2e18a6d-9313-4b12-8109-61a4d930b9e3") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:49:07.225504 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.225292    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs podName:cf3fd93c-f29a-4477-88a7-620311730106 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:39.225283138 +0000 UTC m=+65.225949220 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs") pod "network-metrics-daemon-gjknn" (UID: "cf3fd93c-f29a-4477-88a7-620311730106") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:49:07.325698 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:07.325627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:07.325875 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.325763    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:49:07.325875 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.325788    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:49:07.325875 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.325798    2577 projected.go:196] Error preparing data for projected volume kube-api-access-2pcs5 for pod openshift-network-diagnostics/network-check-target-qg547: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:49:07.325875 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.325855    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5 podName:da5b3772-17b5-4963-bcf5-3872b4c99cd0 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:39.325840809 +0000 UTC m=+65.326506907 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2pcs5" (UniqueName: "kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5") pod "network-check-target-qg547" (UID: "da5b3772-17b5-4963-bcf5-3872b4c99cd0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:49:07.510209 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:07.510131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:07.510355 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:07.510247    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wvkhv" podUID="d2e18a6d-9313-4b12-8109-61a4d930b9e3"
Sep 09 12:49:08.244982 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.244896    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-160.ec2.internal" event="NodeReady"
Sep 09 12:49:08.245557 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.245091    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 12:49:08.285481 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.285443    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7956564d9f-gqskm"]
Sep 09 12:49:08.293894 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.293862    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4jwks"]
Sep 09 12:49:08.294063 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.293992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.296895 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.296873    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qkhdl\""
Sep 09 12:49:08.297037 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.296909    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 12:49:08.297037 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.296954    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 12:49:08.297037 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.296912    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 12:49:08.297655 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.297612    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dqhj7"]
Sep 09 12:49:08.297772 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.297740    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4jwks"
Sep 09 12:49:08.300379 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.300358    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.300514 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.300499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 12:49:08.300869 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.300848    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7956564d9f-gqskm"]
Sep 09 12:49:08.301298 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.300905    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 12:49:08.301519 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.300915    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4hv86\""
Sep 09 12:49:08.301750 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.300971    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 12:49:08.302584 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.302562    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 12:49:08.302786 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.302761    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 12:49:08.302893 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.302849    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s29r7\""
Sep 09 12:49:08.303369 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.303352    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 12:49:08.303447 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.303400    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 12:49:08.303587 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.303574    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 12:49:08.311093 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.311066    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4jwks"]
Sep 09 12:49:08.315166 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.315095    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dqhj7"]
Sep 09 12:49:08.321194 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.321174    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fkxwl"]
Sep 09 12:49:08.324902 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.324885    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.327298 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.327271    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmjpj\""
Sep 09 12:49:08.327439 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.327343    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 12:49:08.327684 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.327666    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 12:49:08.334295 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.334273    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fkxwl"]
Sep 09 12:49:08.432201 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-bound-sa-token\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.432375 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7412b12-e83b-4c9a-a98a-bff21efb9dee-tmp-dir\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.432375 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-certificates\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.432375 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsf6h\" (UniqueName: \"kubernetes.io/projected/c7412b12-e83b-4c9a-a98a-bff21efb9dee-kube-api-access-fsf6h\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.432519 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxdvl\" (UniqueName: \"kubernetes.io/projected/4ee4dc29-0e0c-4f78-93b0-73c254525239-kube-api-access-bxdvl\") pod \"ingress-canary-4jwks\" (UID: \"4ee4dc29-0e0c-4f78-93b0-73c254525239\") " pod="openshift-ingress-canary/ingress-canary-4jwks"
Sep 09 12:49:08.432519 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8x8p\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-kube-api-access-s8x8p\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.432519 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.432622 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/caaa672d-0fed-4fda-a76c-45bf9e027f3e-ca-trust-extracted\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.432622 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-trusted-ca\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.432622 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-crio-socket\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.432622 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.432784 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432649    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-data-volume\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.432784 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-installation-pull-secrets\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.432784 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432691    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.432784 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf5mz\" (UniqueName: \"kubernetes.io/projected/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-api-access-kf5mz\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.432784 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ee4dc29-0e0c-4f78-93b0-73c254525239-cert\") pod \"ingress-canary-4jwks\" (UID: \"4ee4dc29-0e0c-4f78-93b0-73c254525239\") " pod="openshift-ingress-canary/ingress-canary-4jwks"
Sep 09 12:49:08.432784 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.432992 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7412b12-e83b-4c9a-a98a-bff21efb9dee-config-volume\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.432992 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.432837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-image-registry-private-configuration\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.509671 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.509568    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:08.509671 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.509616    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:08.512404 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.512377    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:49:08.512571 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.512427    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ns67g\""
Sep 09 12:49:08.512571 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.512448    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qcx5j\""
Sep 09 12:49:08.512571 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.512460    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:49:08.512571 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.512385    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:49:08.533926 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.533888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-certificates\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.533926 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.533928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsf6h\" (UniqueName: \"kubernetes.io/projected/c7412b12-e83b-4c9a-a98a-bff21efb9dee-kube-api-access-fsf6h\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.534147 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxdvl\" (UniqueName: \"kubernetes.io/projected/4ee4dc29-0e0c-4f78-93b0-73c254525239-kube-api-access-bxdvl\") pod \"ingress-canary-4jwks\" (UID: \"4ee4dc29-0e0c-4f78-93b0-73c254525239\") " pod="openshift-ingress-canary/ingress-canary-4jwks"
Sep 09 12:49:08.534147 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8x8p\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-kube-api-access-s8x8p\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.534147 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.534147 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/caaa672d-0fed-4fda-a76c-45bf9e027f3e-ca-trust-extracted\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.534341 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534233    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:49:08.534341 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-trusted-ca\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.534341 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534300    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:09.034282432 +0000 UTC m=+35.034948511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : configmap "kube-rbac-proxy" not found
Sep 09 12:49:08.534481 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-crio-socket\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.534481 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.534481 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-data-volume\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.534481 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-installation-pull-secrets\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.534481 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf5mz\" (UniqueName: \"kubernetes.io/projected/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-api-access-kf5mz\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534510    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ee4dc29-0e0c-4f78-93b0-73c254525239-cert\") pod \"ingress-canary-4jwks\" (UID: \"4ee4dc29-0e0c-4f78-93b0-73c254525239\") " pod="openshift-ingress-canary/ingress-canary-4jwks"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/caaa672d-0fed-4fda-a76c-45bf9e027f3e-ca-trust-extracted\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534576    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:09.03454925 +0000 UTC m=+35.035215344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7412b12-e83b-4c9a-a98a-bff21efb9dee-config-volume\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534623    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534655    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7956564d9f-gqskm: secret "image-registry-tls" not found
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534670    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-image-registry-private-configuration\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534705    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls podName:caaa672d-0fed-4fda-a76c-45bf9e027f3e nodeName:}" failed. No retries permitted until 2026-09-09 12:49:09.034690387 +0000 UTC m=+35.035356466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls") pod "image-registry-7956564d9f-gqskm" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e") : secret "image-registry-tls" not found
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534711    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-certificates\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-crio-socket\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.534749 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:08.534758    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls podName:c7412b12-e83b-4c9a-a98a-bff21efb9dee nodeName:}" failed. No retries permitted until 2026-09-09 12:49:09.034747933 +0000 UTC m=+35.035414011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls") pod "dns-default-fkxwl" (UID: "c7412b12-e83b-4c9a-a98a-bff21efb9dee") : secret "dns-default-metrics-tls" not found
Sep 09 12:49:08.535428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-bound-sa-token\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.535428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7412b12-e83b-4c9a-a98a-bff21efb9dee-tmp-dir\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.535428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.534988    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-data-volume\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.535428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.535156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7412b12-e83b-4c9a-a98a-bff21efb9dee-tmp-dir\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.535428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.535264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-trusted-ca\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.535428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.535330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c7412b12-e83b-4c9a-a98a-bff21efb9dee-config-volume\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.539664 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.539613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-image-registry-private-configuration\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.539803 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.539611    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4ee4dc29-0e0c-4f78-93b0-73c254525239-cert\") pod \"ingress-canary-4jwks\" (UID: \"4ee4dc29-0e0c-4f78-93b0-73c254525239\") " pod="openshift-ingress-canary/ingress-canary-4jwks"
Sep 09 12:49:08.539803 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.539701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-installation-pull-secrets\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.544531 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.544493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8x8p\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-kube-api-access-s8x8p\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.547031 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.546995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-bound-sa-token\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:08.547142 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.547001    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsf6h\" (UniqueName: \"kubernetes.io/projected/c7412b12-e83b-4c9a-a98a-bff21efb9dee-kube-api-access-fsf6h\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:08.547188 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.547146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxdvl\" (UniqueName: \"kubernetes.io/projected/4ee4dc29-0e0c-4f78-93b0-73c254525239-kube-api-access-bxdvl\") pod \"ingress-canary-4jwks\" (UID: \"4ee4dc29-0e0c-4f78-93b0-73c254525239\") " pod="openshift-ingress-canary/ingress-canary-4jwks"
Sep 09 12:49:08.547706 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.547688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf5mz\" (UniqueName: \"kubernetes.io/projected/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-api-access-kf5mz\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:08.618126 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:08.618083    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4jwks"
Sep 09 12:49:09.039944 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:09.039905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:09.040136 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:09.039960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:09.040136 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:09.039989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:09.040136 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:09.040014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:09.040136 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040032    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:49:09.040136 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040099    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:10.040081001 +0000 UTC m=+36.040747079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : configmap "kube-rbac-proxy" not found
Sep 09 12:49:09.040397 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040143    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:09.040397 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040146    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:49:09.040397 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040151    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:49:09.040397 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040164    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7956564d9f-gqskm: secret "image-registry-tls" not found
Sep 09 12:49:09.040397 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040203    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:10.040187333 +0000 UTC m=+36.040853431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:09.040397 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040223    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls podName:c7412b12-e83b-4c9a-a98a-bff21efb9dee nodeName:}" failed. No retries permitted until 2026-09-09 12:49:10.040213788 +0000 UTC m=+36.040879867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls") pod "dns-default-fkxwl" (UID: "c7412b12-e83b-4c9a-a98a-bff21efb9dee") : secret "dns-default-metrics-tls" not found
Sep 09 12:49:09.040397 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:09.040237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls podName:caaa672d-0fed-4fda-a76c-45bf9e027f3e nodeName:}" failed. No retries permitted until 2026-09-09 12:49:10.040229446 +0000 UTC m=+36.040895523 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls") pod "image-registry-7956564d9f-gqskm" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e") : secret "image-registry-tls" not found
Sep 09 12:49:09.311104 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:09.311017    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dgkg9_482bfe86-6cb5-4c04-a941-e3e81ff24ad6/dns-node-resolver/0.log"
Sep 09 12:49:09.510092 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:09.510053    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:09.512307 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:09.512285    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:49:10.050354 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.050322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.050369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.050393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.050416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050466    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050504    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050512    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050528    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7956564d9f-gqskm: secret "image-registry-tls" not found
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050532    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.050514211 +0000 UTC m=+38.051180290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : configmap "kube-rbac-proxy" not found
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050505    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050565    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.050547765 +0000 UTC m=+38.051213870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050582    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls podName:c7412b12-e83b-4c9a-a98a-bff21efb9dee nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.050573155 +0000 UTC m=+38.051239233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls") pod "dns-default-fkxwl" (UID: "c7412b12-e83b-4c9a-a98a-bff21efb9dee") : secret "dns-default-metrics-tls" not found
Sep 09 12:49:10.050588 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:10.050597    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls podName:caaa672d-0fed-4fda-a76c-45bf9e027f3e nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.050590595 +0000 UTC m=+38.051256673 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls") pod "image-registry-7956564d9f-gqskm" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e") : secret "image-registry-tls" not found
Sep 09 12:49:10.082046 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.082024    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2cjcv_bbad6c3c-19dc-4d5c-a127-7f47350d618a/node-ca/0.log"
Sep 09 12:49:10.414158 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.414131    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4jwks"]
Sep 09 12:49:10.418121 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:10.418095    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ee4dc29_0e0c_4f78_93b0_73c254525239.slice/crio-e481326cb2c3cf516c7af0c800086ed4be2b65401d859d9d3ff6cdaf44d386ad WatchSource:0}: Error finding container e481326cb2c3cf516c7af0c800086ed4be2b65401d859d9d3ff6cdaf44d386ad: Status 404 returned error can't find the container with id e481326cb2c3cf516c7af0c800086ed4be2b65401d859d9d3ff6cdaf44d386ad
Sep 09 12:49:10.722100 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.722060    2577 generic.go:356] "Generic (PLEG): container finished" podID="270732e6-0ab3-42bc-a918-de8d94beeb1a" containerID="f620d9397def035a0796e9135b68beafe5bae73636915bf96f93e1863b6871cb" exitCode=0
Sep 09 12:49:10.722290 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.722153    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerDied","Data":"f620d9397def035a0796e9135b68beafe5bae73636915bf96f93e1863b6871cb"}
Sep 09 12:49:10.723475 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:10.723453    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4jwks" event={"ID":"4ee4dc29-0e0c-4f78-93b0-73c254525239","Type":"ContainerStarted","Data":"e481326cb2c3cf516c7af0c800086ed4be2b65401d859d9d3ff6cdaf44d386ad"}
Sep 09 12:49:11.731114 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:11.731077    2577 generic.go:356] "Generic (PLEG): container finished" podID="270732e6-0ab3-42bc-a918-de8d94beeb1a" containerID="b076aed330b7b9b00053e2209578f59175abf5df2c657da08e189c9f4e35151e" exitCode=0
Sep 09 12:49:11.731819 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:11.731128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerDied","Data":"b076aed330b7b9b00053e2209578f59175abf5df2c657da08e189c9f4e35151e"}
Sep 09 12:49:12.069297 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:12.069207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:12.069297 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069255    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:49:12.069525 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:12.069296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:12.069525 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069323    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:16.069303073 +0000 UTC m=+42.069969159 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : configmap "kube-rbac-proxy" not found
Sep 09 12:49:12.069525 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:12.069352    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:12.069525 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:12.069396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:12.069525 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069417    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:12.069525 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069490    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:16.069464433 +0000 UTC m=+42.070130512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:12.069525 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069500    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:49:12.069525 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069507    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:49:12.069845 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069533    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7956564d9f-gqskm: secret "image-registry-tls" not found
Sep 09 12:49:12.069845 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069562    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls podName:c7412b12-e83b-4c9a-a98a-bff21efb9dee nodeName:}" failed. No retries permitted until 2026-09-09 12:49:16.06954534 +0000 UTC m=+42.070211428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls") pod "dns-default-fkxwl" (UID: "c7412b12-e83b-4c9a-a98a-bff21efb9dee") : secret "dns-default-metrics-tls" not found
Sep 09 12:49:12.069845 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:12.069584    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls podName:caaa672d-0fed-4fda-a76c-45bf9e027f3e nodeName:}" failed. No retries permitted until 2026-09-09 12:49:16.069569083 +0000 UTC m=+42.070235174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls") pod "image-registry-7956564d9f-gqskm" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e") : secret "image-registry-tls" not found
Sep 09 12:49:12.737245 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:12.737208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" event={"ID":"270732e6-0ab3-42bc-a918-de8d94beeb1a","Type":"ContainerStarted","Data":"0d4e4f78f3c07bc9dcd85111f1d5da52cc408640b541aa356c8e334d91d6d602"}
Sep 09 12:49:12.759507 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:12.759408    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xwhz9" podStartSLOduration=7.5905791019999995 podStartE2EDuration="38.759388519s" totalImagesPullingTime="31.168809417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:12.758258743 +0000 UTC m=+38.758924843" watchObservedRunningTime="2026-09-09 12:49:12.759388519 +0000 UTC m=+38.760054619"
Sep 09 12:49:13.741166 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:13.740968    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4jwks" event={"ID":"4ee4dc29-0e0c-4f78-93b0-73c254525239","Type":"ContainerStarted","Data":"cbad554c309fa3171d69883b5a974a4b662a240bdb210d3a0a4bf2c76cd4898b"}
Sep 09 12:49:13.756217 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:13.756160    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4jwks" podStartSLOduration=3.143200425 podStartE2EDuration="5.756140997s" totalImagesPullingTime="2.612940572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:13.755690805 +0000 UTC m=+39.756356905" watchObservedRunningTime="2026-09-09 12:49:13.756140997 +0000 UTC m=+39.756807096"
Sep 09 12:49:16.101904 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:16.101852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:16.101904 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:16.101903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:16.101929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:16.101982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:16.102021    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:16.102084    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls podName:c7412b12-e83b-4c9a-a98a-bff21efb9dee nodeName:}" failed. No retries permitted until 2026-09-09 12:49:24.10206687 +0000 UTC m=+50.102732950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls") pod "dns-default-fkxwl" (UID: "c7412b12-e83b-4c9a-a98a-bff21efb9dee") : secret "dns-default-metrics-tls" not found
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:16.102102    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:16.102021    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:16.102123    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7956564d9f-gqskm: secret "image-registry-tls" not found
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:16.102192    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls podName:caaa672d-0fed-4fda-a76c-45bf9e027f3e nodeName:}" failed. No retries permitted until 2026-09-09 12:49:24.102177825 +0000 UTC m=+50.102843902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls") pod "image-registry-7956564d9f-gqskm" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e") : secret "image-registry-tls" not found
Sep 09 12:49:16.102336 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:16.102239    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls podName:e04d3a42-5b3e-4570-8c33-0f8c54eda3f2 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:24.102218854 +0000 UTC m=+50.102884934 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dqhj7" (UID: "e04d3a42-5b3e-4570-8c33-0f8c54eda3f2") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:16.102602 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:16.102566    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:24.163175 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.163134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:24.163175 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.163187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:24.163713 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.163227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:24.166865 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.166832    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e04d3a42-5b3e-4570-8c33-0f8c54eda3f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dqhj7\" (UID: \"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2\") " pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:24.166994 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.166874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c7412b12-e83b-4c9a-a98a-bff21efb9dee-metrics-tls\") pod \"dns-default-fkxwl\" (UID: \"c7412b12-e83b-4c9a-a98a-bff21efb9dee\") " pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:24.166994 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.166937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"image-registry-7956564d9f-gqskm\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") " pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:24.207943 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.207909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:24.222522 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.222497    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dqhj7"
Sep 09 12:49:24.235051 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.235025    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:24.503072 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.502978    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7956564d9f-gqskm"]
Sep 09 12:49:24.507018 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:24.506988    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaaa672d_0fed_4fda_a76c_45bf9e027f3e.slice/crio-3590c4458f7cba6c7f6efb9cbaa2cfd7d552418ccad2c261fdd1f1af501c5d4c WatchSource:0}: Error finding container 3590c4458f7cba6c7f6efb9cbaa2cfd7d552418ccad2c261fdd1f1af501c5d4c: Status 404 returned error can't find the container with id 3590c4458f7cba6c7f6efb9cbaa2cfd7d552418ccad2c261fdd1f1af501c5d4c
Sep 09 12:49:24.521449 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.521421    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dqhj7"]
Sep 09 12:49:24.524336 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:24.524310    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04d3a42_5b3e_4570_8c33_0f8c54eda3f2.slice/crio-9470893fcf1df4a66deb15a7a6724880fee20d8d524251e5509779d918c984e2 WatchSource:0}: Error finding container 9470893fcf1df4a66deb15a7a6724880fee20d8d524251e5509779d918c984e2: Status 404 returned error can't find the container with id 9470893fcf1df4a66deb15a7a6724880fee20d8d524251e5509779d918c984e2
Sep 09 12:49:24.542775 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.542755    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fkxwl"]
Sep 09 12:49:24.555186 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:24.555163    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7412b12_e83b_4c9a_a98a_bff21efb9dee.slice/crio-059a80ffd43d30581d6b8158e497c9470b8336c2e08d176d615a1c48ec5ff0cc WatchSource:0}: Error finding container 059a80ffd43d30581d6b8158e497c9470b8336c2e08d176d615a1c48ec5ff0cc: Status 404 returned error can't find the container with id 059a80ffd43d30581d6b8158e497c9470b8336c2e08d176d615a1c48ec5ff0cc
Sep 09 12:49:24.769605 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.769507    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fkxwl" event={"ID":"c7412b12-e83b-4c9a-a98a-bff21efb9dee","Type":"ContainerStarted","Data":"059a80ffd43d30581d6b8158e497c9470b8336c2e08d176d615a1c48ec5ff0cc"}
Sep 09 12:49:24.771318 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.771279    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7956564d9f-gqskm" event={"ID":"caaa672d-0fed-4fda-a76c-45bf9e027f3e","Type":"ContainerStarted","Data":"389ff67633bebe6ee758df4a95db6de4aa5ee53ee9411873c6c931a5138949ef"}
Sep 09 12:49:24.771462 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.771327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7956564d9f-gqskm" event={"ID":"caaa672d-0fed-4fda-a76c-45bf9e027f3e","Type":"ContainerStarted","Data":"3590c4458f7cba6c7f6efb9cbaa2cfd7d552418ccad2c261fdd1f1af501c5d4c"}
Sep 09 12:49:24.771462 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.771351    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:24.773058 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.773028    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dqhj7" event={"ID":"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2","Type":"ContainerStarted","Data":"dd2021f31386c42d224f53af6774a2cbfccdb384bfa2105113e00c1840431028"}
Sep 09 12:49:24.773170 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.773075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dqhj7" event={"ID":"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2","Type":"ContainerStarted","Data":"9470893fcf1df4a66deb15a7a6724880fee20d8d524251e5509779d918c984e2"}
Sep 09 12:49:24.790442 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:24.790386    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7956564d9f-gqskm" podStartSLOduration=36.790364554 podStartE2EDuration="36.790364554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:24.789243124 +0000 UTC m=+50.789909226" watchObservedRunningTime="2026-09-09 12:49:24.790364554 +0000 UTC m=+50.791030656"
Sep 09 12:49:26.779437 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:26.779398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fkxwl" event={"ID":"c7412b12-e83b-4c9a-a98a-bff21efb9dee","Type":"ContainerStarted","Data":"bd9142a42c3a72e595ead4f3d82337564f55403376f1f51c943292357ccc49bf"}
Sep 09 12:49:26.779910 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:26.779453    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fkxwl" event={"ID":"c7412b12-e83b-4c9a-a98a-bff21efb9dee","Type":"ContainerStarted","Data":"ce8ddd6e15523283690d17866d9895ef9f04fb1d6074bf5ee6cbfccddc0780f8"}
Sep 09 12:49:26.779910 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:26.779513    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:26.797594 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:26.797542    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fkxwl" podStartSLOduration=17.070086517 podStartE2EDuration="18.797524347s" totalImagesPullingTime="1.72743783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:26.796738707 +0000 UTC m=+52.797404822" watchObservedRunningTime="2026-09-09 12:49:26.797524347 +0000 UTC m=+52.798190446"
Sep 09 12:49:27.788620 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:27.788570    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dqhj7" event={"ID":"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2","Type":"ContainerStarted","Data":"cf28468732d646f6637926271d1b80270124dae1439061ed8dbc60ffa5c39d78"}
Sep 09 12:49:28.794120 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:28.794085    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dqhj7" event={"ID":"e04d3a42-5b3e-4570-8c33-0f8c54eda3f2","Type":"ContainerStarted","Data":"b621fa3e31ca095a7b6774f783e3de438d55d3f497b02e4d9bbfe74d233e62d3"}
Sep 09 12:49:28.810242 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:28.810187    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dqhj7" podStartSLOduration=17.42746826 podStartE2EDuration="20.810171278s" totalImagesPullingTime="3.382703018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:49:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:28.809462365 +0000 UTC m=+54.810128464" watchObservedRunningTime="2026-09-09 12:49:28.810171278 +0000 UTC m=+54.810837402"
Sep 09 12:49:32.718084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.718046    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"]
Sep 09 12:49:32.751566 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.751535    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"]
Sep 09 12:49:32.751748 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.751668    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.754144 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.754119    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 09 12:49:32.754278 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.754120    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 12:49:32.754278 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.754119    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 12:49:32.754278 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.754164    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 12:49:32.765111 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.765090    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"]
Sep 09 12:49:32.765221 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.765132    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"]
Sep 09 12:49:32.765221 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.765210    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.767509 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.767487    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 09 12:49:32.767509 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.767488    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 09 12:49:32.767731 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.767535    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 09 12:49:32.767731 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.767563    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 09 12:49:32.800399 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.800369    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7956564d9f-gqskm"]
Sep 09 12:49:32.830031 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-klusterlet-config\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.830193 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6n26\" (UniqueName: \"kubernetes.io/projected/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-kube-api-access-t6n26\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.830193 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxc7x\" (UniqueName: \"kubernetes.io/projected/6abb6e4a-ae28-4fd8-a549-4b215482bf58-kube-api-access-xxc7x\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.830193 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.830193 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-tmp\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.830193 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-hub\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.830193 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-ca\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.830193 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abb6e4a-ae28-4fd8-a549-4b215482bf58-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.830193 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.830196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.930561 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-klusterlet-config\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.930561 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6n26\" (UniqueName: \"kubernetes.io/projected/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-kube-api-access-t6n26\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.930827 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxc7x\" (UniqueName: \"kubernetes.io/projected/6abb6e4a-ae28-4fd8-a549-4b215482bf58-kube-api-access-xxc7x\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.930827 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.930827 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-tmp\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.930827 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-hub\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.930827 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-ca\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.930827 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abb6e4a-ae28-4fd8-a549-4b215482bf58-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.931174 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.930846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.931230 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.931178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-tmp\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.931684 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.931624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6abb6e4a-ae28-4fd8-a549-4b215482bf58-additional-service-ca\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.933369 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.933346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-ca\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.933907 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.933886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.934010 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.933995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-klusterlet-config\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:32.934073 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.933991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.934151 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.934133    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6abb6e4a-ae28-4fd8-a549-4b215482bf58-hub\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.948623 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.948595    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxc7x\" (UniqueName: \"kubernetes.io/projected/6abb6e4a-ae28-4fd8-a549-4b215482bf58-kube-api-access-xxc7x\") pod \"cluster-proxy-proxy-agent-558c496c8d-thw87\" (UID: \"6abb6e4a-ae28-4fd8-a549-4b215482bf58\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:32.948816 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:32.948798    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6n26\" (UniqueName: \"kubernetes.io/projected/7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1-kube-api-access-t6n26\") pod \"klusterlet-addon-workmgr-f48f64985-8kg4r\" (UID: \"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:33.062005 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:33.061909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:33.087100 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:33.087065    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"
Sep 09 12:49:33.372977 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:33.372896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"]
Sep 09 12:49:33.376160 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:33.376127    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7204a8e4_e6b7_42e2_9d0e_ff4b9f4ddce1.slice/crio-e7f4464cc28d89cded3308c21f64df67db31bb7d6f3305e6ceb6de67eb790098 WatchSource:0}: Error finding container e7f4464cc28d89cded3308c21f64df67db31bb7d6f3305e6ceb6de67eb790098: Status 404 returned error can't find the container with id e7f4464cc28d89cded3308c21f64df67db31bb7d6f3305e6ceb6de67eb790098
Sep 09 12:49:33.397614 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:33.397587    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87"]
Sep 09 12:49:33.400508 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:33.400480    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6abb6e4a_ae28_4fd8_a549_4b215482bf58.slice/crio-34274cc3c4064fb49b1c25c31e290b30e47ee7305efe49b1a6b3e378a0bac74d WatchSource:0}: Error finding container 34274cc3c4064fb49b1c25c31e290b30e47ee7305efe49b1a6b3e378a0bac74d: Status 404 returned error can't find the container with id 34274cc3c4064fb49b1c25c31e290b30e47ee7305efe49b1a6b3e378a0bac74d
Sep 09 12:49:33.810806 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:33.810757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r" event={"ID":"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1","Type":"ContainerStarted","Data":"e7f4464cc28d89cded3308c21f64df67db31bb7d6f3305e6ceb6de67eb790098"}
Sep 09 12:49:33.811864 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:33.811843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87" event={"ID":"6abb6e4a-ae28-4fd8-a549-4b215482bf58","Type":"ContainerStarted","Data":"34274cc3c4064fb49b1c25c31e290b30e47ee7305efe49b1a6b3e378a0bac74d"}
Sep 09 12:49:34.879443 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:34.879400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-djzps"
Sep 09 12:49:36.791095 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:36.791060    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fkxwl"
Sep 09 12:49:38.835988 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:38.835942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r" event={"ID":"7204a8e4-e6b7-42e2-9d0e-ff4b9f4ddce1","Type":"ContainerStarted","Data":"f318f13373100a697b3e21b85f9ee8d2d8bbe743ad3500a76adce41cac859eb5"}
Sep 09 12:49:38.836381 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:38.836154    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:38.837966 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:38.837942    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r"
Sep 09 12:49:38.852100 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:38.852055    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f48f64985-8kg4r" podStartSLOduration=2.325618891 podStartE2EDuration="6.852040431s" totalImagesPullingTime="4.52642154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:38.851175861 +0000 UTC m=+64.851841973" watchObservedRunningTime="2026-09-09 12:49:38.852040431 +0000 UTC m=+64.852706530"
Sep 09 12:49:39.280355 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.280298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:39.280577 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.280369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:39.283226 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.283203    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:49:39.283327 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.283273    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:49:39.293498 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.293472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2e18a6d-9313-4b12-8109-61a4d930b9e3-original-pull-secret\") pod \"global-pull-secret-syncer-wvkhv\" (UID: \"d2e18a6d-9313-4b12-8109-61a4d930b9e3\") " pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:39.293598 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.293501    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf3fd93c-f29a-4477-88a7-620311730106-metrics-certs\") pod \"network-metrics-daemon-gjknn\" (UID: \"cf3fd93c-f29a-4477-88a7-620311730106\") " pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:39.381051 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.381011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:39.383622 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.383597    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:49:39.394003 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.393983    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:49:39.404428 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.404407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pcs5\" (UniqueName: \"kubernetes.io/projected/da5b3772-17b5-4963-bcf5-3872b4c99cd0-kube-api-access-2pcs5\") pod \"network-check-target-qg547\" (UID: \"da5b3772-17b5-4963-bcf5-3872b4c99cd0\") " pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:39.423114 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.423084    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ns67g\""
Sep 09 12:49:39.426904 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.426886    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qcx5j\""
Sep 09 12:49:39.431565 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.431548    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gjknn"
Sep 09 12:49:39.435074 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.435059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:39.520764 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.520728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wvkhv"
Sep 09 12:49:39.711890 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.711859    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gjknn"]
Sep 09 12:49:39.714121 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.714093    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qg547"]
Sep 09 12:49:39.715153 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:39.715129    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3fd93c_f29a_4477_88a7_620311730106.slice/crio-6863e331b62f67abb02ace48040d84311dd0ae5bb36d8d4d7221d63ec8d50b16 WatchSource:0}: Error finding container 6863e331b62f67abb02ace48040d84311dd0ae5bb36d8d4d7221d63ec8d50b16: Status 404 returned error can't find the container with id 6863e331b62f67abb02ace48040d84311dd0ae5bb36d8d4d7221d63ec8d50b16
Sep 09 12:49:39.716436 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:39.716413    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda5b3772_17b5_4963_bcf5_3872b4c99cd0.slice/crio-4211b548291b639551f022cfbcfe0750bf7fef54729bbecdf444aebb184abeb0 WatchSource:0}: Error finding container 4211b548291b639551f022cfbcfe0750bf7fef54729bbecdf444aebb184abeb0: Status 404 returned error can't find the container with id 4211b548291b639551f022cfbcfe0750bf7fef54729bbecdf444aebb184abeb0
Sep 09 12:49:39.786801 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.786725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wvkhv"]
Sep 09 12:49:39.789785 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:39.789741    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e18a6d_9313_4b12_8109_61a4d930b9e3.slice/crio-202c5c68408242981de2e5c9bcfe3c6caec91bf8fa0bde12613072a0037d6265 WatchSource:0}: Error finding container 202c5c68408242981de2e5c9bcfe3c6caec91bf8fa0bde12613072a0037d6265: Status 404 returned error can't find the container with id 202c5c68408242981de2e5c9bcfe3c6caec91bf8fa0bde12613072a0037d6265
Sep 09 12:49:39.840781 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.840744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wvkhv" event={"ID":"d2e18a6d-9313-4b12-8109-61a4d930b9e3","Type":"ContainerStarted","Data":"202c5c68408242981de2e5c9bcfe3c6caec91bf8fa0bde12613072a0037d6265"}
Sep 09 12:49:39.842085 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.842059    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gjknn" event={"ID":"cf3fd93c-f29a-4477-88a7-620311730106","Type":"ContainerStarted","Data":"6863e331b62f67abb02ace48040d84311dd0ae5bb36d8d4d7221d63ec8d50b16"}
Sep 09 12:49:39.843207 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:39.843190    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qg547" event={"ID":"da5b3772-17b5-4963-bcf5-3872b4c99cd0","Type":"ContainerStarted","Data":"4211b548291b639551f022cfbcfe0750bf7fef54729bbecdf444aebb184abeb0"}
Sep 09 12:49:41.855540 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:41.854958    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gjknn" event={"ID":"cf3fd93c-f29a-4477-88a7-620311730106","Type":"ContainerStarted","Data":"4882a29680018f06cb3d813c8c54af38f9ca60f59a0beff534ffb5a138403ae0"}
Sep 09 12:49:41.855540 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:41.855029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gjknn" event={"ID":"cf3fd93c-f29a-4477-88a7-620311730106","Type":"ContainerStarted","Data":"953688de425fcc3f665c76c16b0fdfbff81ffd68a35b5490fbf8ba15f8d2d704"}
Sep 09 12:49:41.871591 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:41.871526    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gjknn" podStartSLOduration=66.099400923 podStartE2EDuration="1m7.871502004s" totalImagesPullingTime="1.772101081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:41.870030659 +0000 UTC m=+67.870696760" watchObservedRunningTime="2026-09-09 12:49:41.871502004 +0000 UTC m=+67.872168108"
Sep 09 12:49:42.808629 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:42.808584    2577 patch_prober.go:28] interesting pod/image-registry-7956564d9f-gqskm container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 09 12:49:42.808857 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:42.808693    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7956564d9f-gqskm" podUID="caaa672d-0fed-4fda-a76c-45bf9e027f3e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:49:44.871806 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:44.871395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wvkhv" event={"ID":"d2e18a6d-9313-4b12-8109-61a4d930b9e3","Type":"ContainerStarted","Data":"3fb96dacd0eeb18dedd2f87c9c2fa53ac67455e1c4ae80e34e56572608b3e8b1"}
Sep 09 12:49:44.874187 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:44.874143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qg547" event={"ID":"da5b3772-17b5-4963-bcf5-3872b4c99cd0","Type":"ContainerStarted","Data":"96a6222367989c3c62a3d320897f8270052efa4a63e088c2280895637b235a7a"}
Sep 09 12:49:44.874342 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:44.874228    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:49:44.888865 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:44.887756    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wvkhv" podStartSLOduration=66.06399512 podStartE2EDuration="1m10.887734781s" totalImagesPullingTime="4.823739661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:44.886531456 +0000 UTC m=+70.887197558" watchObservedRunningTime="2026-09-09 12:49:44.887734781 +0000 UTC m=+70.888400882"
Sep 09 12:49:44.900947 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:44.900362    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qg547" podStartSLOduration=66.007982769 podStartE2EDuration="1m10.900312591s" totalImagesPullingTime="4.892329822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:44.899877225 +0000 UTC m=+70.900543325" watchObservedRunningTime="2026-09-09 12:49:44.900312591 +0000 UTC m=+70.900978689"
Sep 09 12:49:45.354462 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.354421    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hwg9d"]
Sep 09 12:49:45.377551 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.377515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.380447 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.380413    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 12:49:45.380709 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.380691    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 12:49:45.381265 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.381245    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 12:49:45.383582 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.382115    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 12:49:45.383582 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.382508    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 12:49:45.383582 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.382970    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t4lg7\""
Sep 09 12:49:45.383582 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.383216    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 12:49:45.423935 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.423900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-tls\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.424107 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.423947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.424107 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.423973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-accelerators-collector-config\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.424107 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.424060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-textfile\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.424107 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.424103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-wtmp\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.424281 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.424140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-sys\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.424281 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.424171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgg29\" (UniqueName: \"kubernetes.io/projected/9fa88c30-3155-49b8-a3e6-fce37598f630-kube-api-access-kgg29\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.424281 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.424224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fa88c30-3155-49b8-a3e6-fce37598f630-metrics-client-ca\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.424281 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.424267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-root\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525102 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525276 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-accelerators-collector-config\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525276 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-textfile\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525276 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-wtmp\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525276 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-sys\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525276 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgg29\" (UniqueName: \"kubernetes.io/projected/9fa88c30-3155-49b8-a3e6-fce37598f630-kube-api-access-kgg29\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525276 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fa88c30-3155-49b8-a3e6-fce37598f630-metrics-client-ca\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525590 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-root\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525590 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-tls\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525590 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525398    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-sys\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525590 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:45.525459    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 12:49:45.525590 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:49:45.525543    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-tls podName:9fa88c30-3155-49b8-a3e6-fce37598f630 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:46.025522348 +0000 UTC m=+72.026188445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-tls") pod "node-exporter-hwg9d" (UID: "9fa88c30-3155-49b8-a3e6-fce37598f630") : secret "node-exporter-tls" not found
Sep 09 12:49:45.525590 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525562    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-wtmp\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525921 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-textfile\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525921 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525760    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9fa88c30-3155-49b8-a3e6-fce37598f630-root\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.525921 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.525895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-accelerators-collector-config\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.526084 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.526015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fa88c30-3155-49b8-a3e6-fce37598f630-metrics-client-ca\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.528253 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.528228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:45.534782 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:45.534757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgg29\" (UniqueName: \"kubernetes.io/projected/9fa88c30-3155-49b8-a3e6-fce37598f630-kube-api-access-kgg29\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:46.031060 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:46.031026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-tls\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:46.034025 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:46.033998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9fa88c30-3155-49b8-a3e6-fce37598f630-node-exporter-tls\") pod \"node-exporter-hwg9d\" (UID: \"9fa88c30-3155-49b8-a3e6-fce37598f630\") " pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:46.293280 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:46.293187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hwg9d"
Sep 09 12:49:47.619625 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:47.619600    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa88c30_3155_49b8_a3e6_fce37598f630.slice/crio-50428f8afd4da4a17e41ff87fe8204205abe7c90af8df520381894ecc67c91ec WatchSource:0}: Error finding container 50428f8afd4da4a17e41ff87fe8204205abe7c90af8df520381894ecc67c91ec: Status 404 returned error can't find the container with id 50428f8afd4da4a17e41ff87fe8204205abe7c90af8df520381894ecc67c91ec
Sep 09 12:49:47.887946 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:47.887912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87" event={"ID":"6abb6e4a-ae28-4fd8-a549-4b215482bf58","Type":"ContainerStarted","Data":"d4a3e696d7e1b7d744352826e03965a5d5fed00ccad3b4048825aa0c5a6f5dca"}
Sep 09 12:49:47.888146 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:47.887956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87" event={"ID":"6abb6e4a-ae28-4fd8-a549-4b215482bf58","Type":"ContainerStarted","Data":"8b131a8a4d29c738cc72b40a812cb4ecb158dbc7968aee91af8172e358a77f08"}
Sep 09 12:49:47.888146 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:47.887971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87" event={"ID":"6abb6e4a-ae28-4fd8-a549-4b215482bf58","Type":"ContainerStarted","Data":"64364a02b1c85cf0810504218acf32b6e050e7bafe00bff94222464f8bf1e408"}
Sep 09 12:49:47.889215 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:47.889194    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hwg9d" event={"ID":"9fa88c30-3155-49b8-a3e6-fce37598f630","Type":"ContainerStarted","Data":"50428f8afd4da4a17e41ff87fe8204205abe7c90af8df520381894ecc67c91ec"}
Sep 09 12:49:47.906953 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:47.906897    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-558c496c8d-thw87" podStartSLOduration=1.647669015 podStartE2EDuration="15.906878656s" totalImagesPullingTime="14.259209641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:47.906395073 +0000 UTC m=+73.907061173" watchObservedRunningTime="2026-09-09 12:49:47.906878656 +0000 UTC m=+73.907544755"
Sep 09 12:49:48.894900 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:48.894866    2577 generic.go:356] "Generic (PLEG): container finished" podID="9fa88c30-3155-49b8-a3e6-fce37598f630" containerID="1394041a87b8dc6ee25eeea119a6cd45e130ede7bc4cb5f190fe36ecaf7604b6" exitCode=0
Sep 09 12:49:48.895352 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:48.894947    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hwg9d" event={"ID":"9fa88c30-3155-49b8-a3e6-fce37598f630","Type":"ContainerDied","Data":"1394041a87b8dc6ee25eeea119a6cd45e130ede7bc4cb5f190fe36ecaf7604b6"}
Sep 09 12:49:49.651397 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.651363    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"]
Sep 09 12:49:49.653228 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.653212    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.655540 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.655515    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 09 12:49:49.656721 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.656695    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-chnmsc6kfd3vo\""
Sep 09 12:49:49.656721 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.656715    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 12:49:49.656890 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.656715    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 09 12:49:49.656890 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.656790    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fhzb9\""
Sep 09 12:49:49.656890 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.656698    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 09 12:49:49.662983 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.662962    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"]
Sep 09 12:49:49.763048 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.763014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/77e98066-88f4-43cd-9bff-3e40850d6d1e-audit-log\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.763048 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.763057    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-secret-metrics-server-tls\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.763315 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.763082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-client-ca-bundle\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.763315 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.763156    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/77e98066-88f4-43cd-9bff-3e40850d6d1e-metrics-server-audit-profiles\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.763315 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.763204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77e98066-88f4-43cd-9bff-3e40850d6d1e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.763315 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.763233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz5jw\" (UniqueName: \"kubernetes.io/projected/77e98066-88f4-43cd-9bff-3e40850d6d1e-kube-api-access-bz5jw\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.763315 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.763251    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-secret-metrics-server-client-certs\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.864568 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.864518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/77e98066-88f4-43cd-9bff-3e40850d6d1e-metrics-server-audit-profiles\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.864568 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.864567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77e98066-88f4-43cd-9bff-3e40850d6d1e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.864808 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.864590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz5jw\" (UniqueName: \"kubernetes.io/projected/77e98066-88f4-43cd-9bff-3e40850d6d1e-kube-api-access-bz5jw\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.864808 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.864614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-secret-metrics-server-client-certs\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.864808 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.864682    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/77e98066-88f4-43cd-9bff-3e40850d6d1e-audit-log\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.864808 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.864711    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-secret-metrics-server-tls\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.864981 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.864849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-client-ca-bundle\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.865097 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.865066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/77e98066-88f4-43cd-9bff-3e40850d6d1e-audit-log\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.865390 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.865369    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77e98066-88f4-43cd-9bff-3e40850d6d1e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.865887 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.865864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/77e98066-88f4-43cd-9bff-3e40850d6d1e-metrics-server-audit-profiles\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.867312 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.867290    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-secret-metrics-server-client-certs\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.867415 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.867401    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-client-ca-bundle\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.867480 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.867459    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/77e98066-88f4-43cd-9bff-3e40850d6d1e-secret-metrics-server-tls\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.880598 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.880564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz5jw\" (UniqueName: \"kubernetes.io/projected/77e98066-88f4-43cd-9bff-3e40850d6d1e-kube-api-access-bz5jw\") pod \"metrics-server-5f7d488c4c-n5pfw\" (UID: \"77e98066-88f4-43cd-9bff-3e40850d6d1e\") " pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:49.900476 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.900445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hwg9d" event={"ID":"9fa88c30-3155-49b8-a3e6-fce37598f630","Type":"ContainerStarted","Data":"6515bf6dac636cfc1fe8161613b6fef81d411c051e60ffae3c67323877efd2e0"}
Sep 09 12:49:49.900904 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.900494    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hwg9d" event={"ID":"9fa88c30-3155-49b8-a3e6-fce37598f630","Type":"ContainerStarted","Data":"c9db7dc56ea0c06edbdf31c82b95082761cca7aa754da45b3c73dab6a5ead49c"}
Sep 09 12:49:49.919213 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.919113    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hwg9d" podStartSLOduration=3.871011064 podStartE2EDuration="4.919094271s" totalImagesPullingTime="975.714984ms" totalInitContainerRuntime="72.368223ms" isStatefulPod=true podCreationTimestamp="2026-09-09 12:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:49.918356344 +0000 UTC m=+75.919022471" watchObservedRunningTime="2026-09-09 12:49:49.919094271 +0000 UTC m=+75.919760372"
Sep 09 12:49:49.962586 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:49.962553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:49:50.231936 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:50.231906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"]
Sep 09 12:49:50.234501 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:49:50.234470    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77e98066_88f4_43cd_9bff_3e40850d6d1e.slice/crio-8e8f13e78c2bd7d775f5d104ee8e3416e6637973cc812bae408d8f597540674f WatchSource:0}: Error finding container 8e8f13e78c2bd7d775f5d104ee8e3416e6637973cc812bae408d8f597540674f: Status 404 returned error can't find the container with id 8e8f13e78c2bd7d775f5d104ee8e3416e6637973cc812bae408d8f597540674f
Sep 09 12:49:50.904706 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:50.904671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw" event={"ID":"77e98066-88f4-43cd-9bff-3e40850d6d1e","Type":"ContainerStarted","Data":"8e8f13e78c2bd7d775f5d104ee8e3416e6637973cc812bae408d8f597540674f"}
Sep 09 12:49:52.805453 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:52.805424    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:52.913444 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:52.913345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw" event={"ID":"77e98066-88f4-43cd-9bff-3e40850d6d1e","Type":"ContainerStarted","Data":"887d225bd7dada81c34276fc8fad70b99743e76d7c6fb6ebdd94c2bf41f5f118"}
Sep 09 12:49:52.930190 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:52.930134    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw" podStartSLOduration=1.514871226 podStartE2EDuration="3.930117067s" totalImagesPullingTime="2.415245841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:52.929438752 +0000 UTC m=+78.930104851" watchObservedRunningTime="2026-09-09 12:49:52.930117067 +0000 UTC m=+78.930783197"
Sep 09 12:49:53.534776 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.534743    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56cf49fc-f8kk8"]
Sep 09 12:49:53.538035 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.538018    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.540732 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.540706    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 12:49:53.542405 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.542380    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 12:49:53.542405 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.542400    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 12:49:53.542568 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.542400    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 12:49:53.542568 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.542406    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 12:49:53.542568 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.542473    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 12:49:53.542740 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.542610    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2vjnl\""
Sep 09 12:49:53.542740 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.542733    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 12:49:53.547701 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.547680    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56cf49fc-f8kk8"]
Sep 09 12:49:53.591689 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.591652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-oauth-serving-cert\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.591847 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.591699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-config\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.591847 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.591732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tzlq\" (UniqueName: \"kubernetes.io/projected/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-kube-api-access-8tzlq\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.591847 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.591753    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-serving-cert\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.591847 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.591770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-service-ca\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.591847 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.591820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-oauth-config\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.693009 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.692969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-service-ca\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.693009 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.693013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-oauth-config\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.693228 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.693055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-oauth-serving-cert\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.693228 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.693077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-config\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.693228 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.693110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tzlq\" (UniqueName: \"kubernetes.io/projected/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-kube-api-access-8tzlq\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.693228 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.693140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-serving-cert\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.693797 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.693765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-service-ca\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.694102 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.693806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-oauth-serving-cert\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.694102 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.693820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-config\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.696384 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.696356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-oauth-config\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.696548 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.696528    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-serving-cert\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.701445 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.701419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tzlq\" (UniqueName: \"kubernetes.io/projected/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-kube-api-access-8tzlq\") pod \"console-56cf49fc-f8kk8\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") " pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:53.847330 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:53.847234    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:49:54.119619 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:54.119582    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56cf49fc-f8kk8"]
Sep 09 12:49:54.923033 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:54.922959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cf49fc-f8kk8" event={"ID":"fb611c2d-c8d7-4c8d-83e7-c30b182cd196","Type":"ContainerStarted","Data":"7ab88eba1069d10bd5804ca914fcaa76c39654dda3ab979854f57f4d12b44eaa"}
Sep 09 12:49:56.931907 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:56.931871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cf49fc-f8kk8" event={"ID":"fb611c2d-c8d7-4c8d-83e7-c30b182cd196","Type":"ContainerStarted","Data":"e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c"}
Sep 09 12:49:56.947717 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:56.947661    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56cf49fc-f8kk8" podStartSLOduration=1.2660441869999999 podStartE2EDuration="3.947618742s" totalImagesPullingTime="2.681574555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:56.946736619 +0000 UTC m=+82.947402720" watchObservedRunningTime="2026-09-09 12:49:56.947618742 +0000 UTC m=+82.948284847"
Sep 09 12:49:57.820422 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:57.820350    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7956564d9f-gqskm" podUID="caaa672d-0fed-4fda-a76c-45bf9e027f3e" containerName="registry" containerID="cri-o://389ff67633bebe6ee758df4a95db6de4aa5ee53ee9411873c6c931a5138949ef" gracePeriod=30
Sep 09 12:49:57.940579 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:57.940542    2577 generic.go:356] "Generic (PLEG): container finished" podID="caaa672d-0fed-4fda-a76c-45bf9e027f3e" containerID="389ff67633bebe6ee758df4a95db6de4aa5ee53ee9411873c6c931a5138949ef" exitCode=0
Sep 09 12:49:57.941083 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:57.940622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7956564d9f-gqskm" event={"ID":"caaa672d-0fed-4fda-a76c-45bf9e027f3e","Type":"ContainerDied","Data":"389ff67633bebe6ee758df4a95db6de4aa5ee53ee9411873c6c931a5138949ef"}
Sep 09 12:49:58.062396 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.062372    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:58.126311 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126217    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-certificates\") pod \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") "
Sep 09 12:49:58.126311 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126264    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") pod \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") "
Sep 09 12:49:58.126311 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-image-registry-private-configuration\") pod \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") "
Sep 09 12:49:58.126584 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126322    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8x8p\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-kube-api-access-s8x8p\") pod \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") "
Sep 09 12:49:58.126584 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126393    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-installation-pull-secrets\") pod \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") "
Sep 09 12:49:58.126584 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126427    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-bound-sa-token\") pod \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") "
Sep 09 12:49:58.126584 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126455    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-trusted-ca\") pod \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") "
Sep 09 12:49:58.126584 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126486    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/caaa672d-0fed-4fda-a76c-45bf9e027f3e-ca-trust-extracted\") pod \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\" (UID: \"caaa672d-0fed-4fda-a76c-45bf9e027f3e\") "
Sep 09 12:49:58.126836 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.126679    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "caaa672d-0fed-4fda-a76c-45bf9e027f3e" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:58.127332 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.127297    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "caaa672d-0fed-4fda-a76c-45bf9e027f3e" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:49:58.129068 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.129032    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "caaa672d-0fed-4fda-a76c-45bf9e027f3e" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:49:58.129289 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.129259    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "caaa672d-0fed-4fda-a76c-45bf9e027f3e" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:49:58.129398 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.129311    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-kube-api-access-s8x8p" (OuterVolumeSpecName: "kube-api-access-s8x8p") pod "caaa672d-0fed-4fda-a76c-45bf9e027f3e" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e"). InnerVolumeSpecName "kube-api-access-s8x8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:58.129398 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.129379    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "caaa672d-0fed-4fda-a76c-45bf9e027f3e" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:58.129514 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.129451    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "caaa672d-0fed-4fda-a76c-45bf9e027f3e" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:49:58.135131 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.135109    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caaa672d-0fed-4fda-a76c-45bf9e027f3e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "caaa672d-0fed-4fda-a76c-45bf9e027f3e" (UID: "caaa672d-0fed-4fda-a76c-45bf9e027f3e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:49:58.227834 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.227795    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-installation-pull-secrets\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:49:58.227834 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.227828    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-bound-sa-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:49:58.227834 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.227838    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-trusted-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:49:58.228073 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.227848    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/caaa672d-0fed-4fda-a76c-45bf9e027f3e-ca-trust-extracted\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:49:58.228073 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.227857    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-certificates\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:49:58.228073 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.227865    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-registry-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:49:58.228073 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.227874    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/caaa672d-0fed-4fda-a76c-45bf9e027f3e-image-registry-private-configuration\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:49:58.228073 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.227882    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8x8p\" (UniqueName: \"kubernetes.io/projected/caaa672d-0fed-4fda-a76c-45bf9e027f3e-kube-api-access-s8x8p\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:49:58.945238 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.945205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7956564d9f-gqskm" event={"ID":"caaa672d-0fed-4fda-a76c-45bf9e027f3e","Type":"ContainerDied","Data":"3590c4458f7cba6c7f6efb9cbaa2cfd7d552418ccad2c261fdd1f1af501c5d4c"}
Sep 09 12:49:58.945238 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.945230    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7956564d9f-gqskm"
Sep 09 12:49:58.945747 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.945264    2577 scope.go:117] "RemoveContainer" containerID="389ff67633bebe6ee758df4a95db6de4aa5ee53ee9411873c6c931a5138949ef"
Sep 09 12:49:58.961356 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.961333    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7956564d9f-gqskm"]
Sep 09 12:49:58.964923 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:49:58.964900    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7956564d9f-gqskm"]
Sep 09 12:50:00.514794 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:00.514764    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caaa672d-0fed-4fda-a76c-45bf9e027f3e" path="/var/lib/kubelet/pods/caaa672d-0fed-4fda-a76c-45bf9e027f3e/volumes"
Sep 09 12:50:03.847379 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:03.847344    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:50:03.847784 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:03.847404    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:50:03.852257 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:03.852234    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:50:03.968150 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:03.968119    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:50:09.963208 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:09.963161    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:50:09.963591 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:09.963230    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:50:13.032503 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:13.032467    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56cf49fc-f8kk8"]
Sep 09 12:50:15.881082 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:15.881052    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qg547"
Sep 09 12:50:27.675782 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:27.675752    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4jwks_4ee4dc29-0e0c-4f78-93b0-73c254525239/serve-healthcheck-canary/0.log"
Sep 09 12:50:29.968521 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:29.968492    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:50:29.972494 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:29.972468    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5f7d488c4c-n5pfw"
Sep 09 12:50:38.053530 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.053463    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56cf49fc-f8kk8" podUID="fb611c2d-c8d7-4c8d-83e7-c30b182cd196" containerName="console" containerID="cri-o://e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c" gracePeriod=15
Sep 09 12:50:38.302233 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.302207    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:50:38.427971 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.427936    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-service-ca\") pod \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") "
Sep 09 12:50:38.428159 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.427997    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-oauth-config\") pod \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") "
Sep 09 12:50:38.428159 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.428034    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-config\") pod \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") "
Sep 09 12:50:38.428159 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.428061    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-serving-cert\") pod \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") "
Sep 09 12:50:38.428159 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.428079    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-oauth-serving-cert\") pod \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") "
Sep 09 12:50:38.428159 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.428100    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tzlq\" (UniqueName: \"kubernetes.io/projected/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-kube-api-access-8tzlq\") pod \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\" (UID: \"fb611c2d-c8d7-4c8d-83e7-c30b182cd196\") "
Sep 09 12:50:38.428411 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.428329    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb611c2d-c8d7-4c8d-83e7-c30b182cd196" (UID: "fb611c2d-c8d7-4c8d-83e7-c30b182cd196"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:38.428476 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.428405    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-config" (OuterVolumeSpecName: "console-config") pod "fb611c2d-c8d7-4c8d-83e7-c30b182cd196" (UID: "fb611c2d-c8d7-4c8d-83e7-c30b182cd196"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:38.428544 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.428514    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb611c2d-c8d7-4c8d-83e7-c30b182cd196" (UID: "fb611c2d-c8d7-4c8d-83e7-c30b182cd196"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:38.430758 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.430728    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-kube-api-access-8tzlq" (OuterVolumeSpecName: "kube-api-access-8tzlq") pod "fb611c2d-c8d7-4c8d-83e7-c30b182cd196" (UID: "fb611c2d-c8d7-4c8d-83e7-c30b182cd196"). InnerVolumeSpecName "kube-api-access-8tzlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:38.430886 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.430796    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb611c2d-c8d7-4c8d-83e7-c30b182cd196" (UID: "fb611c2d-c8d7-4c8d-83e7-c30b182cd196"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:50:38.430996 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.430975    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb611c2d-c8d7-4c8d-83e7-c30b182cd196" (UID: "fb611c2d-c8d7-4c8d-83e7-c30b182cd196"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:50:38.528870 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.528845    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:50:38.528870 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.528869    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-oauth-config\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:50:38.529049 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.528878    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-config\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:50:38.529049 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.528887    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-console-serving-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:50:38.529049 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.528895    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-oauth-serving-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:50:38.529049 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:38.528904    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tzlq\" (UniqueName: \"kubernetes.io/projected/fb611c2d-c8d7-4c8d-83e7-c30b182cd196-kube-api-access-8tzlq\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:50:39.089760 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.088283    2577 generic.go:356] "Generic (PLEG): container finished" podID="fb611c2d-c8d7-4c8d-83e7-c30b182cd196" containerID="e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c" exitCode=0
Sep 09 12:50:39.089760 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.088405    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cf49fc-f8kk8" event={"ID":"fb611c2d-c8d7-4c8d-83e7-c30b182cd196","Type":"ContainerDied","Data":"e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c"}
Sep 09 12:50:39.089760 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.088456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cf49fc-f8kk8" event={"ID":"fb611c2d-c8d7-4c8d-83e7-c30b182cd196","Type":"ContainerDied","Data":"7ab88eba1069d10bd5804ca914fcaa76c39654dda3ab979854f57f4d12b44eaa"}
Sep 09 12:50:39.089760 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.088488    2577 scope.go:117] "RemoveContainer" containerID="e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c"
Sep 09 12:50:39.089760 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.088687    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cf49fc-f8kk8"
Sep 09 12:50:39.100602 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.100581    2577 scope.go:117] "RemoveContainer" containerID="e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c"
Sep 09 12:50:39.100995 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:50:39.100967    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c\": container with ID starting with e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c not found: ID does not exist" containerID="e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c"
Sep 09 12:50:39.101100 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.101005    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c"} err="failed to get container status \"e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c\": rpc error: code = NotFound desc = could not find container \"e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c\": container with ID starting with e05ce2115af75bbd61abb64eeea9fa6a2af9b7a8c496badef4d8db3e39d6c36c not found: ID does not exist"
Sep 09 12:50:39.104180 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.104150    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56cf49fc-f8kk8"]
Sep 09 12:50:39.105790 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:39.105770    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56cf49fc-f8kk8"]
Sep 09 12:50:40.514738 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:50:40.514704    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb611c2d-c8d7-4c8d-83e7-c30b182cd196" path="/var/lib/kubelet/pods/fb611c2d-c8d7-4c8d-83e7-c30b182cd196/volumes"
Sep 09 12:53:34.374723 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:34.374699    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 12:53:36.541914 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.541880    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"]
Sep 09 12:53:36.544497 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.542195    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb611c2d-c8d7-4c8d-83e7-c30b182cd196" containerName="console"
Sep 09 12:53:36.544497 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.542212    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb611c2d-c8d7-4c8d-83e7-c30b182cd196" containerName="console"
Sep 09 12:53:36.544497 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.542226    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caaa672d-0fed-4fda-a76c-45bf9e027f3e" containerName="registry"
Sep 09 12:53:36.544497 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.542235    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="caaa672d-0fed-4fda-a76c-45bf9e027f3e" containerName="registry"
Sep 09 12:53:36.544497 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.542286    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="caaa672d-0fed-4fda-a76c-45bf9e027f3e" containerName="registry"
Sep 09 12:53:36.544497 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.542297    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb611c2d-c8d7-4c8d-83e7-c30b182cd196" containerName="console"
Sep 09 12:53:36.545237 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.545217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.547791 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.547761    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:53:36.547910 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.547838    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:53:36.548678 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.548663    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wclvq\""
Sep 09 12:53:36.554852 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.554834    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"]
Sep 09 12:53:36.607688 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.607661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cghwd\" (UniqueName: \"kubernetes.io/projected/d2f1d6c5-f306-47e7-b95f-36056219909e-kube-api-access-cghwd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.607788 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.607694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.607788 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.607713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.708450 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.708424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cghwd\" (UniqueName: \"kubernetes.io/projected/d2f1d6c5-f306-47e7-b95f-36056219909e-kube-api-access-cghwd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.708620 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.708459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.708620 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.708479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.708913 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.708897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.708951 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.708897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.717190 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.717159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cghwd\" (UniqueName: \"kubernetes.io/projected/d2f1d6c5-f306-47e7-b95f-36056219909e-kube-api-access-cghwd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:36.854546 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:36.854474    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:37.124231 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:37.124202    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"]
Sep 09 12:53:37.126563 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:53:37.126536    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f1d6c5_f306_47e7_b95f_36056219909e.slice/crio-b7302102baa7df9c00301c9712525493633ba979f57ce97562d0eca88dc2511e WatchSource:0}: Error finding container b7302102baa7df9c00301c9712525493633ba979f57ce97562d0eca88dc2511e: Status 404 returned error can't find the container with id b7302102baa7df9c00301c9712525493633ba979f57ce97562d0eca88dc2511e
Sep 09 12:53:37.128713 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:37.128697    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:53:37.672555 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:37.672516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8" event={"ID":"d2f1d6c5-f306-47e7-b95f-36056219909e","Type":"ContainerStarted","Data":"b7302102baa7df9c00301c9712525493633ba979f57ce97562d0eca88dc2511e"}
Sep 09 12:53:43.695818 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:43.695777    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerID="77bea78f4cbbd93a5e9428973ab4b9b9fb2682bcc1816066c022bda8468d1d4d" exitCode=0
Sep 09 12:53:43.696272 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:43.695866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8" event={"ID":"d2f1d6c5-f306-47e7-b95f-36056219909e","Type":"ContainerDied","Data":"77bea78f4cbbd93a5e9428973ab4b9b9fb2682bcc1816066c022bda8468d1d4d"}
Sep 09 12:53:46.707413 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:46.707377    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerID="d3ac26feb4b0b22601a081c34f87cb33153c4f37bb128a3d46c793c030a67a4d" exitCode=0
Sep 09 12:53:46.707803 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:46.707418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8" event={"ID":"d2f1d6c5-f306-47e7-b95f-36056219909e","Type":"ContainerDied","Data":"d3ac26feb4b0b22601a081c34f87cb33153c4f37bb128a3d46c793c030a67a4d"}
Sep 09 12:53:53.737884 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:53.737840    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerID="e16ba695d5e021dc37c220285fd83fec13b18067c850efac3450412616634299" exitCode=0
Sep 09 12:53:53.738287 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:53.737910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8" event={"ID":"d2f1d6c5-f306-47e7-b95f-36056219909e","Type":"ContainerDied","Data":"e16ba695d5e021dc37c220285fd83fec13b18067c850efac3450412616634299"}
Sep 09 12:53:54.859514 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:54.859492    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:54.950748 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:54.950726    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-util\") pod \"d2f1d6c5-f306-47e7-b95f-36056219909e\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") "
Sep 09 12:53:54.950874 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:54.950757    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cghwd\" (UniqueName: \"kubernetes.io/projected/d2f1d6c5-f306-47e7-b95f-36056219909e-kube-api-access-cghwd\") pod \"d2f1d6c5-f306-47e7-b95f-36056219909e\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") "
Sep 09 12:53:54.950874 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:54.950805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-bundle\") pod \"d2f1d6c5-f306-47e7-b95f-36056219909e\" (UID: \"d2f1d6c5-f306-47e7-b95f-36056219909e\") "
Sep 09 12:53:54.951506 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:54.951483    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-bundle" (OuterVolumeSpecName: "bundle") pod "d2f1d6c5-f306-47e7-b95f-36056219909e" (UID: "d2f1d6c5-f306-47e7-b95f-36056219909e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:53:54.953163 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:54.953139    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2f1d6c5-f306-47e7-b95f-36056219909e-kube-api-access-cghwd" (OuterVolumeSpecName: "kube-api-access-cghwd") pod "d2f1d6c5-f306-47e7-b95f-36056219909e" (UID: "d2f1d6c5-f306-47e7-b95f-36056219909e"). InnerVolumeSpecName "kube-api-access-cghwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:53:54.955000 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:54.954977    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-util" (OuterVolumeSpecName: "util") pod "d2f1d6c5-f306-47e7-b95f-36056219909e" (UID: "d2f1d6c5-f306-47e7-b95f-36056219909e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:53:55.052148 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:55.052095    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-util\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:53:55.052148 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:55.052117    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cghwd\" (UniqueName: \"kubernetes.io/projected/d2f1d6c5-f306-47e7-b95f-36056219909e-kube-api-access-cghwd\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:53:55.052148 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:55.052128    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d2f1d6c5-f306-47e7-b95f-36056219909e-bundle\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:53:55.745313 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:55.745268    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8" event={"ID":"d2f1d6c5-f306-47e7-b95f-36056219909e","Type":"ContainerDied","Data":"b7302102baa7df9c00301c9712525493633ba979f57ce97562d0eca88dc2511e"}
Sep 09 12:53:55.745313 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:55.745312    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675404w8g8"
Sep 09 12:53:55.745513 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:53:55.745332    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7302102baa7df9c00301c9712525493633ba979f57ce97562d0eca88dc2511e"
Sep 09 12:54:06.284276 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.284239    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"]
Sep 09 12:54:06.284616 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.284491    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerName="util"
Sep 09 12:54:06.284616 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.284506    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerName="util"
Sep 09 12:54:06.284616 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.284526    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerName="pull"
Sep 09 12:54:06.284616 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.284532    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerName="pull"
Sep 09 12:54:06.284616 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.284537    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerName="extract"
Sep 09 12:54:06.284616 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.284542    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerName="extract"
Sep 09 12:54:06.284616 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.284585    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2f1d6c5-f306-47e7-b95f-36056219909e" containerName="extract"
Sep 09 12:54:06.287036 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.287021    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"
Sep 09 12:54:06.289645 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.289608    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xdj57\""
Sep 09 12:54:06.290485 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.290461    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 12:54:06.290485 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.290480    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 12:54:06.296310 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.296284    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"]
Sep 09 12:54:06.425548 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.425518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28faa277-8b69-4c5c-b163-fffc8270263a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fct8f\" (UID: \"28faa277-8b69-4c5c-b163-fffc8270263a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"
Sep 09 12:54:06.425681 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.425565    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prdf9\" (UniqueName: \"kubernetes.io/projected/28faa277-8b69-4c5c-b163-fffc8270263a-kube-api-access-prdf9\") pod \"cert-manager-cainjector-559cdd8c87-fct8f\" (UID: \"28faa277-8b69-4c5c-b163-fffc8270263a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"
Sep 09 12:54:06.526492 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.526468    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prdf9\" (UniqueName: \"kubernetes.io/projected/28faa277-8b69-4c5c-b163-fffc8270263a-kube-api-access-prdf9\") pod \"cert-manager-cainjector-559cdd8c87-fct8f\" (UID: \"28faa277-8b69-4c5c-b163-fffc8270263a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"
Sep 09 12:54:06.526586 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.526508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28faa277-8b69-4c5c-b163-fffc8270263a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fct8f\" (UID: \"28faa277-8b69-4c5c-b163-fffc8270263a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"
Sep 09 12:54:06.534659 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.534581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/28faa277-8b69-4c5c-b163-fffc8270263a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-fct8f\" (UID: \"28faa277-8b69-4c5c-b163-fffc8270263a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"
Sep 09 12:54:06.534807 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.534788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prdf9\" (UniqueName: \"kubernetes.io/projected/28faa277-8b69-4c5c-b163-fffc8270263a-kube-api-access-prdf9\") pod \"cert-manager-cainjector-559cdd8c87-fct8f\" (UID: \"28faa277-8b69-4c5c-b163-fffc8270263a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"
Sep 09 12:54:06.596619 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.596599    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"
Sep 09 12:54:06.862847 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:06.862671    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-fct8f"]
Sep 09 12:54:06.865337 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:54:06.865311    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28faa277_8b69_4c5c_b163_fffc8270263a.slice/crio-2d049c1af8d0fb86c845572275c7464ac52792601055808d43bcde2b4eb581f8 WatchSource:0}: Error finding container 2d049c1af8d0fb86c845572275c7464ac52792601055808d43bcde2b4eb581f8: Status 404 returned error can't find the container with id 2d049c1af8d0fb86c845572275c7464ac52792601055808d43bcde2b4eb581f8
Sep 09 12:54:07.794493 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:07.794449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f" event={"ID":"28faa277-8b69-4c5c-b163-fffc8270263a","Type":"ContainerStarted","Data":"2d049c1af8d0fb86c845572275c7464ac52792601055808d43bcde2b4eb581f8"}
Sep 09 12:54:09.802803 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:09.802767    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f" event={"ID":"28faa277-8b69-4c5c-b163-fffc8270263a","Type":"ContainerStarted","Data":"7c3a63c9ad10caeb0ff4964c4150c15e82b117c28f1a742138cf792eab2cb759"}
Sep 09 12:54:09.820401 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:09.820350    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-fct8f" podStartSLOduration=1.361234665 podStartE2EDuration="3.82033436s" totalImagesPullingTime="2.459099695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:54:09.819504837 +0000 UTC m=+335.820170934" watchObservedRunningTime="2026-09-09 12:54:09.82033436 +0000 UTC m=+335.821000459"
Sep 09 12:54:47.572685 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.572614    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"]
Sep 09 12:54:47.574882 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.574867    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.577168 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.577145    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:54:47.578031 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.578004    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:54:47.578031 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.578007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wclvq\""
Sep 09 12:54:47.583124 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.583106    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"]
Sep 09 12:54:47.590011 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.589985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7n4r\" (UniqueName: \"kubernetes.io/projected/b0648bc1-9c4a-4f15-9ac3-c7297141d572-kube-api-access-b7n4r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.590108 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.590016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.590149 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.590115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.691105 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.691079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.691201 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.691113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7n4r\" (UniqueName: \"kubernetes.io/projected/b0648bc1-9c4a-4f15-9ac3-c7297141d572-kube-api-access-b7n4r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.691201 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.691134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.691431 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.691415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.691495 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.691451    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.698990 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.698972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7n4r\" (UniqueName: \"kubernetes.io/projected/b0648bc1-9c4a-4f15-9ac3-c7297141d572-kube-api-access-b7n4r\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:47.884372 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:47.884349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:48.151819 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:48.151729    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"]
Sep 09 12:54:48.154267 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:54:48.154243    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0648bc1_9c4a_4f15_9ac3_c7297141d572.slice/crio-85939581e512fb036f3a17334431aa8ad18b29ffe2848145999e831ba752dfe2 WatchSource:0}: Error finding container 85939581e512fb036f3a17334431aa8ad18b29ffe2848145999e831ba752dfe2: Status 404 returned error can't find the container with id 85939581e512fb036f3a17334431aa8ad18b29ffe2848145999e831ba752dfe2
Sep 09 12:54:48.941703 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:48.941671    2577 generic.go:356] "Generic (PLEG): container finished" podID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerID="76f7f5843a359a5920bf9b0875bd9e4ee9e2819d76e22f2fb260dd6dac4dc916" exitCode=0
Sep 09 12:54:48.941703 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:48.941707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg" event={"ID":"b0648bc1-9c4a-4f15-9ac3-c7297141d572","Type":"ContainerDied","Data":"76f7f5843a359a5920bf9b0875bd9e4ee9e2819d76e22f2fb260dd6dac4dc916"}
Sep 09 12:54:48.942078 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:48.941741    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg" event={"ID":"b0648bc1-9c4a-4f15-9ac3-c7297141d572","Type":"ContainerStarted","Data":"85939581e512fb036f3a17334431aa8ad18b29ffe2848145999e831ba752dfe2"}
Sep 09 12:54:50.952202 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:50.952166    2577 generic.go:356] "Generic (PLEG): container finished" podID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerID="16e42f19db6dfd77815ada260c6642977aba829527a2e7e42158e6f27dbc6222" exitCode=0
Sep 09 12:54:50.952568 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:50.952217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg" event={"ID":"b0648bc1-9c4a-4f15-9ac3-c7297141d572","Type":"ContainerDied","Data":"16e42f19db6dfd77815ada260c6642977aba829527a2e7e42158e6f27dbc6222"}
Sep 09 12:54:51.957792 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:51.957760    2577 generic.go:356] "Generic (PLEG): container finished" podID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerID="b1e2a3853e8c780320651946d025e7a39579c667593eaceaf9ae53776edd4a14" exitCode=0
Sep 09 12:54:51.958239 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:51.957809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg" event={"ID":"b0648bc1-9c4a-4f15-9ac3-c7297141d572","Type":"ContainerDied","Data":"b1e2a3853e8c780320651946d025e7a39579c667593eaceaf9ae53776edd4a14"}
Sep 09 12:54:52.193000 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.192967    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"]
Sep 09 12:54:52.195908 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.195886    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.202664 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.198891    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-g9kqz\""
Sep 09 12:54:52.202664 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.199149    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:54:52.202664 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.199152    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 09 12:54:52.202664 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.199360    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 09 12:54:52.202664 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.199365    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 09 12:54:52.202664 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.199667    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 12:54:52.209347 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.209298    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"]
Sep 09 12:54:52.220030 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.220010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bee787c5-ea08-40f4-a217-36256d1bfff7-metrics-cert\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.220114 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.220043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j6qt\" (UniqueName: \"kubernetes.io/projected/bee787c5-ea08-40f4-a217-36256d1bfff7-kube-api-access-4j6qt\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.220114 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.220070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bee787c5-ea08-40f4-a217-36256d1bfff7-cert\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.220191 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.220127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bee787c5-ea08-40f4-a217-36256d1bfff7-manager-config\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.320629 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.320600    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bee787c5-ea08-40f4-a217-36256d1bfff7-cert\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.320629 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.320630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bee787c5-ea08-40f4-a217-36256d1bfff7-manager-config\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.320814 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.320677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bee787c5-ea08-40f4-a217-36256d1bfff7-metrics-cert\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.320814 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.320703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j6qt\" (UniqueName: \"kubernetes.io/projected/bee787c5-ea08-40f4-a217-36256d1bfff7-kube-api-access-4j6qt\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.321358 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.321326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bee787c5-ea08-40f4-a217-36256d1bfff7-manager-config\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.323123 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.323096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bee787c5-ea08-40f4-a217-36256d1bfff7-cert\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.323223 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.323188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bee787c5-ea08-40f4-a217-36256d1bfff7-metrics-cert\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.329769 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.329747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j6qt\" (UniqueName: \"kubernetes.io/projected/bee787c5-ea08-40f4-a217-36256d1bfff7-kube-api-access-4j6qt\") pod \"lws-controller-manager-798b96659f-nwsxc\" (UID: \"bee787c5-ea08-40f4-a217-36256d1bfff7\") " pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.507449 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.507393    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:54:52.777937 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.777790    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"]
Sep 09 12:54:52.780271 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:54:52.780242    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee787c5_ea08_40f4_a217_36256d1bfff7.slice/crio-0a149999568aefb37d434f4b7ad2684006a625a2e6da5a809892dff3842a8216 WatchSource:0}: Error finding container 0a149999568aefb37d434f4b7ad2684006a625a2e6da5a809892dff3842a8216: Status 404 returned error can't find the container with id 0a149999568aefb37d434f4b7ad2684006a625a2e6da5a809892dff3842a8216
Sep 09 12:54:52.963214 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:52.963175    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc" event={"ID":"bee787c5-ea08-40f4-a217-36256d1bfff7","Type":"ContainerStarted","Data":"0a149999568aefb37d434f4b7ad2684006a625a2e6da5a809892dff3842a8216"}
Sep 09 12:54:53.074980 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.074957    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:53.126556 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.126530    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7n4r\" (UniqueName: \"kubernetes.io/projected/b0648bc1-9c4a-4f15-9ac3-c7297141d572-kube-api-access-b7n4r\") pod \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") "
Sep 09 12:54:53.126709 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.126601    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-util\") pod \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") "
Sep 09 12:54:53.126709 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.126676    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-bundle\") pod \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\" (UID: \"b0648bc1-9c4a-4f15-9ac3-c7297141d572\") "
Sep 09 12:54:53.127342 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.127314    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-bundle" (OuterVolumeSpecName: "bundle") pod "b0648bc1-9c4a-4f15-9ac3-c7297141d572" (UID: "b0648bc1-9c4a-4f15-9ac3-c7297141d572"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:54:53.128711 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.128690    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0648bc1-9c4a-4f15-9ac3-c7297141d572-kube-api-access-b7n4r" (OuterVolumeSpecName: "kube-api-access-b7n4r") pod "b0648bc1-9c4a-4f15-9ac3-c7297141d572" (UID: "b0648bc1-9c4a-4f15-9ac3-c7297141d572"). InnerVolumeSpecName "kube-api-access-b7n4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:54:53.132415 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.132393    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-util" (OuterVolumeSpecName: "util") pod "b0648bc1-9c4a-4f15-9ac3-c7297141d572" (UID: "b0648bc1-9c4a-4f15-9ac3-c7297141d572"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:54:53.227088 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.227065    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-util\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:54:53.227088 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.227084    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0648bc1-9c4a-4f15-9ac3-c7297141d572-bundle\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:54:53.227201 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.227094    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7n4r\" (UniqueName: \"kubernetes.io/projected/b0648bc1-9c4a-4f15-9ac3-c7297141d572-kube-api-access-b7n4r\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 12:54:53.968176 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.968141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg" event={"ID":"b0648bc1-9c4a-4f15-9ac3-c7297141d572","Type":"ContainerDied","Data":"85939581e512fb036f3a17334431aa8ad18b29ffe2848145999e831ba752dfe2"}
Sep 09 12:54:53.968176 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.968158    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnl8fg"
Sep 09 12:54:53.968560 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:54:53.968185    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85939581e512fb036f3a17334431aa8ad18b29ffe2848145999e831ba752dfe2"
Sep 09 12:55:04.010077 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.010049    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"]
Sep 09 12:55:04.010421 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.010261    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerName="extract"
Sep 09 12:55:04.010421 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.010271    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerName="extract"
Sep 09 12:55:04.010421 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.010282    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerName="pull"
Sep 09 12:55:04.010421 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.010287    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerName="pull"
Sep 09 12:55:04.010421 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.010294    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerName="util"
Sep 09 12:55:04.010421 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.010299    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerName="util"
Sep 09 12:55:04.010421 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.010340    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0648bc1-9c4a-4f15-9ac3-c7297141d572" containerName="extract"
Sep 09 12:55:04.013126 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.013111    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.015928 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.015907    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vtl8s\""
Sep 09 12:55:04.015928 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.015907    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 12:55:04.016405 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.016388    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 12:55:04.016614 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.016599    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 12:55:04.016709 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.016602    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 12:55:04.041621 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.041592    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"]
Sep 09 12:55:04.093656 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.093609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-webhook-cert\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.093748 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.093685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-manifests\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.093748 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.093712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-tmp\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.093748 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.093733    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.093892 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.093799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2wz2\" (UniqueName: \"kubernetes.io/projected/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-kube-api-access-k2wz2\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.194983 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.194955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-manifests\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.195096 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.194986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-tmp\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.195096 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.195002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.195096 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.195028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2wz2\" (UniqueName: \"kubernetes.io/projected/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-kube-api-access-k2wz2\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.195096 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.195058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-webhook-cert\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.195378 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.195359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-manifests\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.197363 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.197342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-tmp\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.197681 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.197665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.197721 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.197705    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-webhook-cert\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.205543 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.205525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2wz2\" (UniqueName: \"kubernetes.io/projected/f8b024cc-fea3-4dcc-b7d7-73b48fc94288-kube-api-access-k2wz2\") pod \"opendatahub-operator-controller-manager-647774d94-9jldd\" (UID: \"f8b024cc-fea3-4dcc-b7d7-73b48fc94288\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.326557 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.326514    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:04.602332 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:04.602307    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"]
Sep 09 12:55:04.605112 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:55:04.605081    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b024cc_fea3_4dcc_b7d7_73b48fc94288.slice/crio-1c8cb00ba84aa325f73c91716a96a35ea13e278884a6ac7b23da5dd4f5bce7f8 WatchSource:0}: Error finding container 1c8cb00ba84aa325f73c91716a96a35ea13e278884a6ac7b23da5dd4f5bce7f8: Status 404 returned error can't find the container with id 1c8cb00ba84aa325f73c91716a96a35ea13e278884a6ac7b23da5dd4f5bce7f8
Sep 09 12:55:05.006604 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:05.006575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd" event={"ID":"f8b024cc-fea3-4dcc-b7d7-73b48fc94288","Type":"ContainerStarted","Data":"1c8cb00ba84aa325f73c91716a96a35ea13e278884a6ac7b23da5dd4f5bce7f8"}
Sep 09 12:55:09.025692 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:09.025656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc" event={"ID":"bee787c5-ea08-40f4-a217-36256d1bfff7","Type":"ContainerStarted","Data":"dd68b172bde26ceebd411165b4fcff5f22ba84ae84aeb942509f69c0b9b1b6c4"}
Sep 09 12:55:09.026150 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:09.025752    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:55:09.027330 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:09.027303    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd" event={"ID":"f8b024cc-fea3-4dcc-b7d7-73b48fc94288","Type":"ContainerStarted","Data":"cf7f56d58bf43999afa7d49baf9f45e979584c7e219b575c9d7a3daa4d091325"}
Sep 09 12:55:09.042571 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:09.042527    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc" podStartSLOduration=1.199146786 podStartE2EDuration="17.042511607s" totalImagesPullingTime="15.843364821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:09.041341487 +0000 UTC m=+395.042007769" watchObservedRunningTime="2026-09-09 12:55:09.042511607 +0000 UTC m=+395.043177706"
Sep 09 12:55:10.034731 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:10.034653    2577 generic.go:356] "Generic (PLEG): container finished" podID="f8b024cc-fea3-4dcc-b7d7-73b48fc94288" containerID="cf7f56d58bf43999afa7d49baf9f45e979584c7e219b575c9d7a3daa4d091325" exitCode=0
Sep 09 12:55:10.034731 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:10.034712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd" event={"ID":"f8b024cc-fea3-4dcc-b7d7-73b48fc94288","Type":"ContainerDied","Data":"cf7f56d58bf43999afa7d49baf9f45e979584c7e219b575c9d7a3daa4d091325"}
Sep 09 12:55:11.040571 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:11.040531    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd" event={"ID":"f8b024cc-fea3-4dcc-b7d7-73b48fc94288","Type":"ContainerStarted","Data":"f8131a9db1397216ed4a3c899c5bf3353e226d6e0844ad9e2575f6f8a243450e"}
Sep 09 12:55:11.040929 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:11.040681    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:11.061459 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:11.061408    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd" podStartSLOduration=2.73938345 podStartE2EDuration="8.061392491s" totalImagesPullingTime="4.335504844s" totalInitContainerRuntime="986.504197ms" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:11.059394686 +0000 UTC m=+397.060060788" watchObservedRunningTime="2026-09-09 12:55:11.061392491 +0000 UTC m=+397.062058599"
Sep 09 12:55:20.037606 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:20.037577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-798b96659f-nwsxc"
Sep 09 12:55:22.046613 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:22.046585    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-9jldd"
Sep 09 12:55:31.742893 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.742861    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"]
Sep 09 12:55:31.745787 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.745768    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.748386 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.748341    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-w5p6z\""
Sep 09 12:55:31.748386 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.748365    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 12:55:31.748386 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.748345    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 12:55:31.748605 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.748397    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 12:55:31.757500 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.757477    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"]
Sep 09 12:55:31.790954 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.790929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.791056 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.790961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.791056 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.790976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.791056 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.790995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.791182 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.791071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p8w9\" (UniqueName: \"kubernetes.io/projected/5d270b03-da4f-49e1-932c-5bf1304ee88e-kube-api-access-7p8w9\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.791182 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.791098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.791182 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.791116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.791182 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.791132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d270b03-da4f-49e1-932c-5bf1304ee88e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.791182 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.791150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892063 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892208 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p8w9\" (UniqueName: \"kubernetes.io/projected/5d270b03-da4f-49e1-932c-5bf1304ee88e-kube-api-access-7p8w9\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892208 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892208 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892208 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892156    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d270b03-da4f-49e1-932c-5bf1304ee88e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892208 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892444 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892444 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892444 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892583 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892583 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892583 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892816 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d270b03-da4f-49e1-932c-5bf1304ee88e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.892919 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.892898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.894773 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.894745    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.894895 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.894881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.901417 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.901390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p8w9\" (UniqueName: \"kubernetes.io/projected/5d270b03-da4f-49e1-932c-5bf1304ee88e-kube-api-access-7p8w9\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:31.901568 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:31.901549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d270b03-da4f-49e1-932c-5bf1304ee88e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cb5blq\" (UID: \"5d270b03-da4f-49e1-932c-5bf1304ee88e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:32.055548 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:32.055475    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:32.333780 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:32.333739    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"]
Sep 09 12:55:32.336199 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:55:32.336172    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d270b03_da4f_49e1_932c_5bf1304ee88e.slice/crio-10ed9453efacbf9c3d92d28c990d48b93789ac6166bd4cf2307663faafe80ea1 WatchSource:0}: Error finding container 10ed9453efacbf9c3d92d28c990d48b93789ac6166bd4cf2307663faafe80ea1: Status 404 returned error can't find the container with id 10ed9453efacbf9c3d92d28c990d48b93789ac6166bd4cf2307663faafe80ea1
Sep 09 12:55:33.120121 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:33.120093    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq" event={"ID":"5d270b03-da4f-49e1-932c-5bf1304ee88e","Type":"ContainerStarted","Data":"10ed9453efacbf9c3d92d28c990d48b93789ac6166bd4cf2307663faafe80ea1"}
Sep 09 12:55:34.981197 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:34.981159    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:55:34.981494 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:34.981228    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:55:34.981494 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:34.981256    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:55:35.130033 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.129997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq" event={"ID":"5d270b03-da4f-49e1-932c-5bf1304ee88e","Type":"ContainerStarted","Data":"38a3ce0754e4243e944478c295b5ce3d0e61c92cb7e7ae35bb13b453e87b694f"}
Sep 09 12:55:35.155944 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.155886    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq" podStartSLOduration=1.5130127230000001 podStartE2EDuration="4.15586644s" totalImagesPullingTime="2.642853717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:35.153524975 +0000 UTC m=+421.154191074" watchObservedRunningTime="2026-09-09 12:55:35.15586644 +0000 UTC m=+421.156532543"
Sep 09 12:55:35.159569 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.159541    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j7lmg"]
Sep 09 12:55:35.162661 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.162630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:35.165412 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.165387    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 09 12:55:35.165521 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.165456    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tp2rp\""
Sep 09 12:55:35.176985 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.176958    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j7lmg"]
Sep 09 12:55:35.217790 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.217762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/110e7393-0f60-4b1a-b58f-62f216970c24-cert\") pod \"odh-model-controller-7998c67dc4-j7lmg\" (UID: \"110e7393-0f60-4b1a-b58f-62f216970c24\") " pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:35.217982 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.217964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzpv7\" (UniqueName: \"kubernetes.io/projected/110e7393-0f60-4b1a-b58f-62f216970c24-kube-api-access-jzpv7\") pod \"odh-model-controller-7998c67dc4-j7lmg\" (UID: \"110e7393-0f60-4b1a-b58f-62f216970c24\") " pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:35.318765 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.318697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/110e7393-0f60-4b1a-b58f-62f216970c24-cert\") pod \"odh-model-controller-7998c67dc4-j7lmg\" (UID: \"110e7393-0f60-4b1a-b58f-62f216970c24\") " pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:35.318765 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.318738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpv7\" (UniqueName: \"kubernetes.io/projected/110e7393-0f60-4b1a-b58f-62f216970c24-kube-api-access-jzpv7\") pod \"odh-model-controller-7998c67dc4-j7lmg\" (UID: \"110e7393-0f60-4b1a-b58f-62f216970c24\") " pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:35.318913 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:55:35.318827    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 09 12:55:35.318913 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:55:35.318882    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/110e7393-0f60-4b1a-b58f-62f216970c24-cert podName:110e7393-0f60-4b1a-b58f-62f216970c24 nodeName:}" failed. No retries permitted until 2026-09-09 12:55:35.818867709 +0000 UTC m=+421.819533786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/110e7393-0f60-4b1a-b58f-62f216970c24-cert") pod "odh-model-controller-7998c67dc4-j7lmg" (UID: "110e7393-0f60-4b1a-b58f-62f216970c24") : secret "odh-model-controller-webhook-cert" not found
Sep 09 12:55:35.326962 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.326933    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzpv7\" (UniqueName: \"kubernetes.io/projected/110e7393-0f60-4b1a-b58f-62f216970c24-kube-api-access-jzpv7\") pod \"odh-model-controller-7998c67dc4-j7lmg\" (UID: \"110e7393-0f60-4b1a-b58f-62f216970c24\") " pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:35.567491 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.567466    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pcl9t"]
Sep 09 12:55:35.570665 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.570615    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:35.572947 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.572922    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 09 12:55:35.572947 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.572935    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-nnnfn\""
Sep 09 12:55:35.578882 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.578860    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pcl9t"]
Sep 09 12:55:35.621028 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.621001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae1f072f-f462-4b0e-b81e-4624c675bad9-tls-certs\") pod \"model-serving-api-54d89bd5fc-pcl9t\" (UID: \"ae1f072f-f462-4b0e-b81e-4624c675bad9\") " pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:35.621113 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.621035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw524\" (UniqueName: \"kubernetes.io/projected/ae1f072f-f462-4b0e-b81e-4624c675bad9-kube-api-access-nw524\") pod \"model-serving-api-54d89bd5fc-pcl9t\" (UID: \"ae1f072f-f462-4b0e-b81e-4624c675bad9\") " pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:35.721791 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.721762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae1f072f-f462-4b0e-b81e-4624c675bad9-tls-certs\") pod \"model-serving-api-54d89bd5fc-pcl9t\" (UID: \"ae1f072f-f462-4b0e-b81e-4624c675bad9\") " pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:35.721915 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.721804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw524\" (UniqueName: \"kubernetes.io/projected/ae1f072f-f462-4b0e-b81e-4624c675bad9-kube-api-access-nw524\") pod \"model-serving-api-54d89bd5fc-pcl9t\" (UID: \"ae1f072f-f462-4b0e-b81e-4624c675bad9\") " pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:35.721915 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:55:35.721910    2577 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 09 12:55:35.721995 ip-10-0-132-160 kubenswrapper[2577]: E0909 12:55:35.721964    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae1f072f-f462-4b0e-b81e-4624c675bad9-tls-certs podName:ae1f072f-f462-4b0e-b81e-4624c675bad9 nodeName:}" failed. No retries permitted until 2026-09-09 12:55:36.221950431 +0000 UTC m=+422.222616508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ae1f072f-f462-4b0e-b81e-4624c675bad9-tls-certs") pod "model-serving-api-54d89bd5fc-pcl9t" (UID: "ae1f072f-f462-4b0e-b81e-4624c675bad9") : secret "model-serving-api-tls" not found
Sep 09 12:55:35.731621 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.731594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw524\" (UniqueName: \"kubernetes.io/projected/ae1f072f-f462-4b0e-b81e-4624c675bad9-kube-api-access-nw524\") pod \"model-serving-api-54d89bd5fc-pcl9t\" (UID: \"ae1f072f-f462-4b0e-b81e-4624c675bad9\") " pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:35.823056 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.822999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/110e7393-0f60-4b1a-b58f-62f216970c24-cert\") pod \"odh-model-controller-7998c67dc4-j7lmg\" (UID: \"110e7393-0f60-4b1a-b58f-62f216970c24\") " pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:35.825320 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:35.825298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/110e7393-0f60-4b1a-b58f-62f216970c24-cert\") pod \"odh-model-controller-7998c67dc4-j7lmg\" (UID: \"110e7393-0f60-4b1a-b58f-62f216970c24\") " pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:36.055951 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.055927    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:36.060463 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.060445    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:36.074279 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.074230    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:36.134737 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.134708    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:36.136791 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.136540    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cb5blq"
Sep 09 12:55:36.226880 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.226849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae1f072f-f462-4b0e-b81e-4624c675bad9-tls-certs\") pod \"model-serving-api-54d89bd5fc-pcl9t\" (UID: \"ae1f072f-f462-4b0e-b81e-4624c675bad9\") " pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:36.229303 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.229283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae1f072f-f462-4b0e-b81e-4624c675bad9-tls-certs\") pod \"model-serving-api-54d89bd5fc-pcl9t\" (UID: \"ae1f072f-f462-4b0e-b81e-4624c675bad9\") " pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:36.344381 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.344358    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j7lmg"]
Sep 09 12:55:36.346807 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:55:36.346778    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110e7393_0f60_4b1a_b58f_62f216970c24.slice/crio-538352df53e5fd7d52a70efe9cc2ec18340ba996509e0775939e86dd61b80f9d WatchSource:0}: Error finding container 538352df53e5fd7d52a70efe9cc2ec18340ba996509e0775939e86dd61b80f9d: Status 404 returned error can't find the container with id 538352df53e5fd7d52a70efe9cc2ec18340ba996509e0775939e86dd61b80f9d
Sep 09 12:55:36.480728 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.480708    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:36.747914 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:36.747886    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-pcl9t"]
Sep 09 12:55:36.750389 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:55:36.750365    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1f072f_f462_4b0e_b81e_4624c675bad9.slice/crio-0213bc0bec84548b1c8170c769d6572ff77019c8805341a7b3c270bb33ba59fb WatchSource:0}: Error finding container 0213bc0bec84548b1c8170c769d6572ff77019c8805341a7b3c270bb33ba59fb: Status 404 returned error can't find the container with id 0213bc0bec84548b1c8170c769d6572ff77019c8805341a7b3c270bb33ba59fb
Sep 09 12:55:37.143834 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:37.143798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t" event={"ID":"ae1f072f-f462-4b0e-b81e-4624c675bad9","Type":"ContainerStarted","Data":"0213bc0bec84548b1c8170c769d6572ff77019c8805341a7b3c270bb33ba59fb"}
Sep 09 12:55:37.145431 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:37.145354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg" event={"ID":"110e7393-0f60-4b1a-b58f-62f216970c24","Type":"ContainerStarted","Data":"538352df53e5fd7d52a70efe9cc2ec18340ba996509e0775939e86dd61b80f9d"}
Sep 09 12:55:40.160973 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:40.160930    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg" event={"ID":"110e7393-0f60-4b1a-b58f-62f216970c24","Type":"ContainerStarted","Data":"2f5141186dcc0b5623c4a203ec05f35c3d3d194c898c5189410b38db23784961"}
Sep 09 12:55:40.161368 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:40.161047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:40.162465 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:40.162434    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t" event={"ID":"ae1f072f-f462-4b0e-b81e-4624c675bad9","Type":"ContainerStarted","Data":"d3a15280e813187462a2487ba26cf705fb9a1992e29ac47df46283607f5fdf46"}
Sep 09 12:55:40.162562 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:40.162490    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:55:40.176333 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:40.176291    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg" podStartSLOduration=1.538899711 podStartE2EDuration="5.176275982s" totalImagesPullingTime="3.637376271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:40.17498165 +0000 UTC m=+426.175647748" watchObservedRunningTime="2026-09-09 12:55:40.176275982 +0000 UTC m=+426.176942080"
Sep 09 12:55:40.190507 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:40.190373    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t" podStartSLOduration=1.919675932 podStartE2EDuration="5.190355892s" totalImagesPullingTime="3.27067996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:40.189791926 +0000 UTC m=+426.190458025" watchObservedRunningTime="2026-09-09 12:55:40.190355892 +0000 UTC m=+426.191021992"
Sep 09 12:55:41.940712 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:41.940676    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"]
Sep 09 12:55:41.943821 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:41.943805    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:41.946462 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:41.946435    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 12:55:41.946462 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:41.946451    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rmv6s\""
Sep 09 12:55:41.946665 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:41.946467    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 12:55:41.946665 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:41.946470    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 12:55:41.957028 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:41.957008    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"]
Sep 09 12:55:42.073494 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.073457    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2e3c5e-df61-41b0-b145-a1c4d245f844-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.073699 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.073500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.073699 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.073529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.073699 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.073605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.073699 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.073663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.073699 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.073695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5ldc\" (UniqueName: \"kubernetes.io/projected/3d2e3c5e-df61-41b0-b145-a1c4d245f844-kube-api-access-l5ldc\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.073916 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.073722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.174831 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.174800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.174831 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.174845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5ldc\" (UniqueName: \"kubernetes.io/projected/3d2e3c5e-df61-41b0-b145-a1c4d245f844-kube-api-access-l5ldc\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.175081 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.174873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.175081 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.174925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2e3c5e-df61-41b0-b145-a1c4d245f844-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.175081 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.174952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.175081 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.174982    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.175280 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.175108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.176029 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.176001    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.177550 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.177520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.177679 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.177662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.177729 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.177662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d2e3c5e-df61-41b0-b145-a1c4d245f844-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.177729 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.177667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3d2e3c5e-df61-41b0-b145-a1c4d245f844-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.183026 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.183000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d2e3c5e-df61-41b0-b145-a1c4d245f844-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.183605 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.183589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5ldc\" (UniqueName: \"kubernetes.io/projected/3d2e3c5e-df61-41b0-b145-a1c4d245f844-kube-api-access-l5ldc\") pod \"istiod-openshift-gateway-5677ccb49b-hdh5c\" (UID: \"3d2e3c5e-df61-41b0-b145-a1c4d245f844\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.252896 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.252811    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:42.529919 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:42.529891    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"]
Sep 09 12:55:43.176315 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:43.176277    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c" event={"ID":"3d2e3c5e-df61-41b0-b145-a1c4d245f844","Type":"ContainerStarted","Data":"ce1aa73100be49872b80adfcb79b25586343da2373dcf8dcb033f003b7667e7f"}
Sep 09 12:55:45.108125 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:45.108084    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:55:45.108373 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:45.108165    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:55:45.186878 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:45.186848    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c" event={"ID":"3d2e3c5e-df61-41b0-b145-a1c4d245f844","Type":"ContainerStarted","Data":"a92cdf6d2efb35ee49d1aab299d9f840e20ffefb0b3cc051615863a5e6219fda"}
Sep 09 12:55:45.187015 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:45.186960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:45.219264 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:45.219212    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c" podStartSLOduration=1.646101925 podStartE2EDuration="4.219195106s" totalImagesPullingTime="2.573093181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:45.219111203 +0000 UTC m=+431.219777303" watchObservedRunningTime="2026-09-09 12:55:45.219195106 +0000 UTC m=+431.219861206"
Sep 09 12:55:46.193328 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:46.193298    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-hdh5c container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:55:46.193727 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:46.193354    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c" podUID="3d2e3c5e-df61-41b0-b145-a1c4d245f844" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:55:49.192589 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:49.192561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-hdh5c"
Sep 09 12:55:51.169486 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:51.169454    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-j7lmg"
Sep 09 12:55:51.171548 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:55:51.171528    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-pcl9t"
Sep 09 12:57:49.552761 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.552673    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"]
Sep 09 12:57:49.556081 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.556059    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.561825 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.561803    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gtgsc\""
Sep 09 12:57:49.580231 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.580208    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"]
Sep 09 12:57:49.710200 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.710352 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.710352 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.710352 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.710352 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.710352 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.710511 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.710511 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710400    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb2f5\" (UniqueName: \"kubernetes.io/projected/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-kube-api-access-vb2f5\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.710511 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.710426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811035 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.810970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811035 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb2f5\" (UniqueName: \"kubernetes.io/projected/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-kube-api-access-vb2f5\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811218 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811218 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811218 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811218 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811218 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811218 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811484 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811672 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811672 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811602    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811672 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.811841 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.811709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.812271 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.812249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.813527 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.813506    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.813783 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.813768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.829922 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.829893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb2f5\" (UniqueName: \"kubernetes.io/projected/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-kube-api-access-vb2f5\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.830146 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.830130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5bfd0059-dde3-45f8-8570-612cfdd5a1fc-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-rfh8m\" (UID: \"5bfd0059-dde3-45f8-8570-612cfdd5a1fc\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:49.865331 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:49.865311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:50.142786 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:50.142761    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"]
Sep 09 12:57:50.145307 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:57:50.145285    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bfd0059_dde3_45f8_8570_612cfdd5a1fc.slice/crio-5c2a387a2d516f4bddb3b8b36e6d8543f1390c0deca7c68c9a3544c30425fcaa WatchSource:0}: Error finding container 5c2a387a2d516f4bddb3b8b36e6d8543f1390c0deca7c68c9a3544c30425fcaa: Status 404 returned error can't find the container with id 5c2a387a2d516f4bddb3b8b36e6d8543f1390c0deca7c68c9a3544c30425fcaa
Sep 09 12:57:50.147226 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:50.147195    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:57:50.147305 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:50.147259    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:57:50.147341 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:50.147306    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:57:50.694069 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:50.694039    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" event={"ID":"5bfd0059-dde3-45f8-8570-612cfdd5a1fc","Type":"ContainerStarted","Data":"661d6123c6ed5836302e3fc324ae968ab563dd2086a14924ee22ad46d0e50e18"}
Sep 09 12:57:50.694453 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:50.694088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" event={"ID":"5bfd0059-dde3-45f8-8570-612cfdd5a1fc","Type":"ContainerStarted","Data":"5c2a387a2d516f4bddb3b8b36e6d8543f1390c0deca7c68c9a3544c30425fcaa"}
Sep 09 12:57:50.720337 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:50.720280    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" podStartSLOduration=1.720263747 podStartE2EDuration="1.720263747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:57:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:57:50.718231139 +0000 UTC m=+556.718897240" watchObservedRunningTime="2026-09-09 12:57:50.720263747 +0000 UTC m=+556.720929846"
Sep 09 12:57:50.866038 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:50.866015    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:51.870281 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:51.870249    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:52.702798 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:52.702767    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:57:52.703963 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:57:52.703942    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:59:22.004399 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.004364    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"]
Sep 09 12:59:22.007644 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.007618    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:22.010107 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.010085    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jghq8\""
Sep 09 12:59:22.010191 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.010122    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 12:59:22.014289 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.013962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7368e50b-ec89-4339-a32c-599551b9e8a2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gm2kv\" (UID: \"7368e50b-ec89-4339-a32c-599551b9e8a2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:22.014289 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.014083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwh6t\" (UniqueName: \"kubernetes.io/projected/7368e50b-ec89-4339-a32c-599551b9e8a2-kube-api-access-wwh6t\") pod \"payload-pre-processing-59fbc47bf8-gm2kv\" (UID: \"7368e50b-ec89-4339-a32c-599551b9e8a2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:22.025844 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.025822    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"]
Sep 09 12:59:22.114613 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.114580    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwh6t\" (UniqueName: \"kubernetes.io/projected/7368e50b-ec89-4339-a32c-599551b9e8a2-kube-api-access-wwh6t\") pod \"payload-pre-processing-59fbc47bf8-gm2kv\" (UID: \"7368e50b-ec89-4339-a32c-599551b9e8a2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:22.114805 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.114623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7368e50b-ec89-4339-a32c-599551b9e8a2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gm2kv\" (UID: \"7368e50b-ec89-4339-a32c-599551b9e8a2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:22.115360 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.115336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7368e50b-ec89-4339-a32c-599551b9e8a2-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-gm2kv\" (UID: \"7368e50b-ec89-4339-a32c-599551b9e8a2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:22.122601 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.122576    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwh6t\" (UniqueName: \"kubernetes.io/projected/7368e50b-ec89-4339-a32c-599551b9e8a2-kube-api-access-wwh6t\") pod \"payload-pre-processing-59fbc47bf8-gm2kv\" (UID: \"7368e50b-ec89-4339-a32c-599551b9e8a2\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:22.317559 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.317500    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:22.587586 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.587508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"]
Sep 09 12:59:22.590116 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:59:22.590090    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7368e50b_ec89_4339_a32c_599551b9e8a2.slice/crio-10b420d0340031552a54cc5ee0f2681c191a6132f52224c80ea36bfec02b4278 WatchSource:0}: Error finding container 10b420d0340031552a54cc5ee0f2681c191a6132f52224c80ea36bfec02b4278: Status 404 returned error can't find the container with id 10b420d0340031552a54cc5ee0f2681c191a6132f52224c80ea36bfec02b4278
Sep 09 12:59:22.592078 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:22.592059    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:59:23.082764 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:23.082723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv" event={"ID":"7368e50b-ec89-4339-a32c-599551b9e8a2","Type":"ContainerStarted","Data":"10b420d0340031552a54cc5ee0f2681c191a6132f52224c80ea36bfec02b4278"}
Sep 09 12:59:25.102886 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:25.102843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv" event={"ID":"7368e50b-ec89-4339-a32c-599551b9e8a2","Type":"ContainerStarted","Data":"15f3a712ebcfdf1ad6cab40a07190dd7454a0054f9934c42f2d79f85ae157462"}
Sep 09 12:59:25.103361 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:25.102943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:25.120534 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:25.120486    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv" podStartSLOduration=2.146531582 podStartE2EDuration="4.120473035s" totalImagesPullingTime="1.973941453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:59:25.118497066 +0000 UTC m=+651.119163166" watchObservedRunningTime="2026-09-09 12:59:25.120473035 +0000 UTC m=+651.121139135"
Sep 09 12:59:32.140035 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:32.139997    2577 generic.go:356] "Generic (PLEG): container finished" podID="5bfd0059-dde3-45f8-8570-612cfdd5a1fc" containerID="661d6123c6ed5836302e3fc324ae968ab563dd2086a14924ee22ad46d0e50e18" exitCode=0
Sep 09 12:59:32.140543 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:32.140073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" event={"ID":"5bfd0059-dde3-45f8-8570-612cfdd5a1fc","Type":"ContainerDied","Data":"661d6123c6ed5836302e3fc324ae968ab563dd2086a14924ee22ad46d0e50e18"}
Sep 09 12:59:32.140543 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:32.140530    2577 scope.go:117] "RemoveContainer" containerID="661d6123c6ed5836302e3fc324ae968ab563dd2086a14924ee22ad46d0e50e18"
Sep 09 12:59:32.141569 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:32.141541    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:59:32.141690 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:32.141608    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:59:32.141690 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:32.141667    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 12:59:33.145718 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:33.145685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" event={"ID":"5bfd0059-dde3-45f8-8570-612cfdd5a1fc","Type":"ContainerStarted","Data":"e1442a52213cd8669edab287301cb113cb920115e3dd587abc1efc9c9f64e13e"}
Sep 09 12:59:33.865654 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:33.865612    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:59:33.866739 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:33.866708    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rfh8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 09 12:59:33.866974 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:33.866944    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" podUID="5bfd0059-dde3-45f8-8570-612cfdd5a1fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 09 12:59:34.866284 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:34.866251    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rfh8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 09 12:59:34.866819 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:34.866346    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" podUID="5bfd0059-dde3-45f8-8570-612cfdd5a1fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 09 12:59:35.866260 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:35.866227    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rfh8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 09 12:59:35.866448 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:35.866283    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" podUID="5bfd0059-dde3-45f8-8570-612cfdd5a1fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 09 12:59:36.109079 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:36.109053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-gm2kv"
Sep 09 12:59:36.866181 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:36.866146    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rfh8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 09 12:59:36.866349 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:36.866205    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" podUID="5bfd0059-dde3-45f8-8570-612cfdd5a1fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 09 12:59:37.705791 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:37.705753    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:59:37.866582 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:37.866546    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-rfh8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Sep 09 12:59:37.866805 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:37.866609    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m" podUID="5bfd0059-dde3-45f8-8570-612cfdd5a1fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Sep 09 12:59:38.869516 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:38.869486    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:59:38.870603 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:38.870584    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-rfh8m"
Sep 09 12:59:55.366805 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.366772    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"]
Sep 09 12:59:55.369710 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.369692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.372508 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.372485    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 12:59:55.372659 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.372485    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pjccl\""
Sep 09 12:59:55.372659 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.372520    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 12:59:55.373403 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.373385    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 09 12:59:55.379277 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.379257    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"]
Sep 09 12:59:55.471716 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.471692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.471829 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.471727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.471829 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.471755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.471829 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.471809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.471928 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.471853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wvwv\" (UniqueName: \"kubernetes.io/projected/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-kube-api-access-2wvwv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.471928 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.471888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.572749 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.572723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.572888 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.572764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.572888 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.572781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.572888 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.572807    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wvwv\" (UniqueName: \"kubernetes.io/projected/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-kube-api-access-2wvwv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.572888 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.572840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.572888 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.572872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.573124 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.573097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.573217 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.573198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.573301 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.573282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.575147 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.575122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.575420 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.575403    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.581482 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.581464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wvwv\" (UniqueName: \"kubernetes.io/projected/6eb7bb09-fd11-4cbe-a834-ad287c5a24d7-kube-api-access-2wvwv\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz\" (UID: \"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.681773 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.681742    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 12:59:55.954903 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:55.954875    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"]
Sep 09 12:59:55.956897 ip-10-0-132-160 kubenswrapper[2577]: W0909 12:59:55.956872    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb7bb09_fd11_4cbe_a834_ad287c5a24d7.slice/crio-0a4b8b7be6425c370b590d84dd538c35ae0158c4ac20b1a1657e792a9239e916 WatchSource:0}: Error finding container 0a4b8b7be6425c370b590d84dd538c35ae0158c4ac20b1a1657e792a9239e916: Status 404 returned error can't find the container with id 0a4b8b7be6425c370b590d84dd538c35ae0158c4ac20b1a1657e792a9239e916
Sep 09 12:59:56.245531 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:56.245429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz" event={"ID":"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7","Type":"ContainerStarted","Data":"0a4b8b7be6425c370b590d84dd538c35ae0158c4ac20b1a1657e792a9239e916"}
Sep 09 12:59:57.250978 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:57.250939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz" event={"ID":"6eb7bb09-fd11-4cbe-a834-ad287c5a24d7","Type":"ContainerStarted","Data":"3e8c38e0bfbb1fc4a2be5ba8ff8365355704cbff377c4c81a7eebda4692f154f"}
Sep 09 12:59:57.268264 ip-10-0-132-160 kubenswrapper[2577]: I0909 12:59:57.268214    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz" podStartSLOduration=1.152379686 podStartE2EDuration="2.268195283s" totalImagesPullingTime="1.115815597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:59:57.267481159 +0000 UTC m=+683.268147260" watchObservedRunningTime="2026-09-09 12:59:57.268195283 +0000 UTC m=+683.268861383"
Sep 09 13:00:05.682811 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:00:05.682776    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 13:00:05.683179 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:00:05.682856    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 13:00:05.694875 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:00:05.694849    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 13:00:06.301426 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:00:06.301393    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz"
Sep 09 13:03:05.848851 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.848818    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"]
Sep 09 13:03:05.851610 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.851587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.853864 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.853843    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3b4b96-openshift-default-dockercfg-n5kmw\""
Sep 09 13:03:05.863210 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.863189    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"]
Sep 09 13:03:05.945787 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.945746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-envoy\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.945991 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.945816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7462m\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-kube-api-access-7462m\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.945991 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.945866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-data\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.945991 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.945951    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-certs\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.946155 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.945999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-socket\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.946155 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.946043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbcbefac-4382-4a7a-93ea-9e5492576609-istiod-ca-cert\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.946155 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.946114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-credential-socket\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.946155 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.946142    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-podinfo\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:05.946353 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:05.946165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-token\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047121 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-token\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047276 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-envoy\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047276 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7462m\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-kube-api-access-7462m\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047276 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-data\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047276 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-certs\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047452 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-socket\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047452 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047385    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbcbefac-4382-4a7a-93ea-9e5492576609-istiod-ca-cert\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047547 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-credential-socket\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047547 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-podinfo\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047680 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047587    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-data\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047747 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-socket\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047801 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-certs\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.047890 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.047869    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-credential-socket\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.048175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.048154    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbcbefac-4382-4a7a-93ea-9e5492576609-istiod-ca-cert\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.049600 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.049578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-envoy\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.049993 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.049975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-podinfo\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.054625 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.054600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-token\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.055615 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.055595    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7462m\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-kube-api-access-7462m\") pod \"e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") " pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.165030 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.165007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:06.440067 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.440008    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"]
Sep 09 13:03:06.442453 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:06.442428    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbcbefac_4382_4a7a_93ea_9e5492576609.slice/crio-4ce787330e29138326a4ad283350bf603fa8d70c744e531a59e2dbc645d16c59 WatchSource:0}: Error finding container 4ce787330e29138326a4ad283350bf603fa8d70c744e531a59e2dbc645d16c59: Status 404 returned error can't find the container with id 4ce787330e29138326a4ad283350bf603fa8d70c744e531a59e2dbc645d16c59
Sep 09 13:03:06.445170 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.445138    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:06.445264 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.445240    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:06.445323 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:06.445286    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:07.065438 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:07.065404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" event={"ID":"cbcbefac-4382-4a7a-93ea-9e5492576609","Type":"ContainerStarted","Data":"b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa"}
Sep 09 13:03:07.065815 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:07.065458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" event={"ID":"cbcbefac-4382-4a7a-93ea-9e5492576609","Type":"ContainerStarted","Data":"4ce787330e29138326a4ad283350bf603fa8d70c744e531a59e2dbc645d16c59"}
Sep 09 13:03:07.101175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:07.101127    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" podStartSLOduration=2.101112126 podStartE2EDuration="2.101112126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:07.099656228 +0000 UTC m=+873.100322320" watchObservedRunningTime="2026-09-09 13:03:07.101112126 +0000 UTC m=+873.101778226"
Sep 09 13:03:07.165932 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:07.165906    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:07.170755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:07.170731    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:08.069870 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:08.069840    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:08.070768 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:08.070747    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:10.559898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.559862    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"]
Sep 09 13:03:10.562203 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.562183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.564903 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.564870    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 13:03:10.564903 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.564896    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 13:03:10.565882 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.565861    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-3b4b96-serving-cert\""
Sep 09 13:03:10.566039 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.565867    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 13:03:10.566977 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.566955    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dbjk4\""
Sep 09 13:03:10.576470 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.576449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"]
Sep 09 13:03:10.688257 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.688226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drlhl\" (UniqueName: \"kubernetes.io/projected/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-kube-api-access-drlhl\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.688434 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.688263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-maas-api-tls\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.688434 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.688298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-metrics-certs\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.688434 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.688393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.789492 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.789435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drlhl\" (UniqueName: \"kubernetes.io/projected/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-kube-api-access-drlhl\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.789684 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.789520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-maas-api-tls\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.789684 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.789575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-metrics-certs\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.789684 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.789651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.790486 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.790457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.792167 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.792144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-metrics-certs\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.792527 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.792510    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-maas-api-tls\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.798370 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.798346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drlhl\" (UniqueName: \"kubernetes.io/projected/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-kube-api-access-drlhl\") pod \"maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:10.876582 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:10.876513    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:11.149677 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.149646    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"]
Sep 09 13:03:11.151664 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:11.151619    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf903cc10_1abf_4f0f_a05b_4d30e5e0c086.slice/crio-6666baf725ef06977547cd7d2a30292a25f8a9a136d56204b77557df2338d050 WatchSource:0}: Error finding container 6666baf725ef06977547cd7d2a30292a25f8a9a136d56204b77557df2338d050: Status 404 returned error can't find the container with id 6666baf725ef06977547cd7d2a30292a25f8a9a136d56204b77557df2338d050
Sep 09 13:03:11.163053 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.163031    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"]
Sep 09 13:03:11.165675 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.165655    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:11.168169 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.168148    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-3b4b96-dockercfg-tmp7z\""
Sep 09 13:03:11.168450 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.168433    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-3b4b96\""
Sep 09 13:03:11.176031 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.176012    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"]
Sep 09 13:03:11.197936 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.197915    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"]
Sep 09 13:03:11.200060 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.200045    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:11.212500 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.212478    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"]
Sep 09 13:03:11.293386 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.293351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c618af0b-148f-4c24-9e05-37c26b9fb268-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w\" (UID: \"c618af0b-148f-4c24-9e05-37c26b9fb268\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:11.293787 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.293717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6wxx\" (UniqueName: \"kubernetes.io/projected/c618af0b-148f-4c24-9e05-37c26b9fb268-kube-api-access-j6wxx\") pod \"payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w\" (UID: \"c618af0b-148f-4c24-9e05-37c26b9fb268\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:11.293908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.293828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c233dc28-13ec-4836-8532-f0c8e48fe1bc-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r\" (UID: \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:11.293908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.293869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg69x\" (UniqueName: \"kubernetes.io/projected/c233dc28-13ec-4836-8532-f0c8e48fe1bc-kube-api-access-cg69x\") pod \"payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r\" (UID: \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:11.394654 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.394608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg69x\" (UniqueName: \"kubernetes.io/projected/c233dc28-13ec-4836-8532-f0c8e48fe1bc-kube-api-access-cg69x\") pod \"payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r\" (UID: \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:11.394887 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.394737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c618af0b-148f-4c24-9e05-37c26b9fb268-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w\" (UID: \"c618af0b-148f-4c24-9e05-37c26b9fb268\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:11.394887 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.394774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6wxx\" (UniqueName: \"kubernetes.io/projected/c618af0b-148f-4c24-9e05-37c26b9fb268-kube-api-access-j6wxx\") pod \"payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w\" (UID: \"c618af0b-148f-4c24-9e05-37c26b9fb268\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:11.394887 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.394839    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c233dc28-13ec-4836-8532-f0c8e48fe1bc-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r\" (UID: \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:11.395484 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.395461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c618af0b-148f-4c24-9e05-37c26b9fb268-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w\" (UID: \"c618af0b-148f-4c24-9e05-37c26b9fb268\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:11.395566 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.395465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c233dc28-13ec-4836-8532-f0c8e48fe1bc-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r\" (UID: \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:11.403548 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.403504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg69x\" (UniqueName: \"kubernetes.io/projected/c233dc28-13ec-4836-8532-f0c8e48fe1bc-kube-api-access-cg69x\") pod \"payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r\" (UID: \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:11.403761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.403741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6wxx\" (UniqueName: \"kubernetes.io/projected/c618af0b-148f-4c24-9e05-37c26b9fb268-kube-api-access-j6wxx\") pod \"payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w\" (UID: \"c618af0b-148f-4c24-9e05-37c26b9fb268\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:11.477416 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.477382    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:11.508983 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.508955    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:11.748700 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.748670    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"]
Sep 09 13:03:11.751166 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:11.751139    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc618af0b_148f_4c24_9e05_37c26b9fb268.slice/crio-c41ce7991bf902554b5fdb38d9bb2dbd47e2ada1117692109cbce6645ffb450f WatchSource:0}: Error finding container c41ce7991bf902554b5fdb38d9bb2dbd47e2ada1117692109cbce6645ffb450f: Status 404 returned error can't find the container with id c41ce7991bf902554b5fdb38d9bb2dbd47e2ada1117692109cbce6645ffb450f
Sep 09 13:03:11.784679 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:11.784652    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"]
Sep 09 13:03:11.786844 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:11.786810    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc233dc28_13ec_4836_8532_f0c8e48fe1bc.slice/crio-581f2cda907d9388caa7d73895dc3bb62cd8298fb96bd61b04c816579ff8fa61 WatchSource:0}: Error finding container 581f2cda907d9388caa7d73895dc3bb62cd8298fb96bd61b04c816579ff8fa61: Status 404 returned error can't find the container with id 581f2cda907d9388caa7d73895dc3bb62cd8298fb96bd61b04c816579ff8fa61
Sep 09 13:03:12.089716 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.089025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r" event={"ID":"c233dc28-13ec-4836-8532-f0c8e48fe1bc","Type":"ContainerStarted","Data":"585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e"}
Sep 09 13:03:12.089716 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.089087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r" event={"ID":"c233dc28-13ec-4836-8532-f0c8e48fe1bc","Type":"ContainerStarted","Data":"581f2cda907d9388caa7d73895dc3bb62cd8298fb96bd61b04c816579ff8fa61"}
Sep 09 13:03:12.089716 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.089689    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:12.094054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.094022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2" event={"ID":"f903cc10-1abf-4f0f-a05b-4d30e5e0c086","Type":"ContainerStarted","Data":"6666baf725ef06977547cd7d2a30292a25f8a9a136d56204b77557df2338d050"}
Sep 09 13:03:12.102935 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.102903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w" event={"ID":"c618af0b-148f-4c24-9e05-37c26b9fb268","Type":"ContainerStarted","Data":"f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1"}
Sep 09 13:03:12.103091 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.102971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w" event={"ID":"c618af0b-148f-4c24-9e05-37c26b9fb268","Type":"ContainerStarted","Data":"c41ce7991bf902554b5fdb38d9bb2dbd47e2ada1117692109cbce6645ffb450f"}
Sep 09 13:03:12.103091 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.103056    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:12.127337 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.127290    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r" podStartSLOduration=1.127274213 podStartE2EDuration="1.127274213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:12.107361707 +0000 UTC m=+878.108027809" watchObservedRunningTime="2026-09-09 13:03:12.127274213 +0000 UTC m=+878.127940310"
Sep 09 13:03:12.128799 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:12.128758    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w" podStartSLOduration=1.128746364 podStartE2EDuration="1.128746364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:12.127077769 +0000 UTC m=+878.127743870" watchObservedRunningTime="2026-09-09 13:03:12.128746364 +0000 UTC m=+878.129412467"
Sep 09 13:03:13.831174 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.831143    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"]
Sep 09 13:03:13.833627 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.833595    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.836022 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.835988    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d0fb1d64-openshift-default-dockercfg-q55d5\""
Sep 09 13:03:13.845447 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.845415    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"]
Sep 09 13:03:13.919800 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.919770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64hr4\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-kube-api-access-64hr4\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.919966 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.919856    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-socket\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.919966 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.919888    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-envoy\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.919966 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.919939    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-data\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.920132 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.919974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-certs\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.920132 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.920050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-credential-socket\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.920132 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.920074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc310562-84e3-40c6-8aa0-b410feccd443-istio-podinfo\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.920132 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.920101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-istio-token\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:13.920324 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:13.920135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc310562-84e3-40c6-8aa0-b410feccd443-istiod-ca-cert\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021332 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-credential-socket\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021347    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc310562-84e3-40c6-8aa0-b410feccd443-istio-podinfo\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-istio-token\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc310562-84e3-40c6-8aa0-b410feccd443-istiod-ca-cert\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021443    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64hr4\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-kube-api-access-64hr4\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021488    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-socket\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-envoy\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021874 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-data\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021874 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-certs\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.021874 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-credential-socket\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.022028 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.021902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-socket\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.022028 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.022015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-data\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.022253 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.022229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-certs\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.022472 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.022456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc310562-84e3-40c6-8aa0-b410feccd443-istiod-ca-cert\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.023956 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.023936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-envoy\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.024065 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.023941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc310562-84e3-40c6-8aa0-b410feccd443-istio-podinfo\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.030024 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.030004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-istio-token\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.030128 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.030023    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64hr4\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-kube-api-access-64hr4\") pod \"e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") " pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.114971 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.114930    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2" event={"ID":"f903cc10-1abf-4f0f-a05b-4d30e5e0c086","Type":"ContainerStarted","Data":"a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981"}
Sep 09 13:03:14.144784 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.144754    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:14.146103 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.146056    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2" podStartSLOduration=1.674506326 podStartE2EDuration="4.1460397s" totalImagesPullingTime="2.471533374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:14.143963917 +0000 UTC m=+880.144630018" watchObservedRunningTime="2026-09-09 13:03:14.1460397 +0000 UTC m=+880.146705802"
Sep 09 13:03:14.417266 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.417210    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"]
Sep 09 13:03:14.419390 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:14.419369    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc310562_84e3_40c6_8aa0_b410feccd443.slice/crio-c2af6e5f44037182ee877358797a830aaac134a18193a8b002e7e914ba25012e WatchSource:0}: Error finding container c2af6e5f44037182ee877358797a830aaac134a18193a8b002e7e914ba25012e: Status 404 returned error can't find the container with id c2af6e5f44037182ee877358797a830aaac134a18193a8b002e7e914ba25012e
Sep 09 13:03:14.421564 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.421533    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:14.421662 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.421594    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:14.421662 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:14.421622    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:15.121518 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:15.121471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" event={"ID":"dc310562-84e3-40c6-8aa0-b410feccd443","Type":"ContainerStarted","Data":"1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473"}
Sep 09 13:03:15.122001 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:15.121546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" event={"ID":"dc310562-84e3-40c6-8aa0-b410feccd443","Type":"ContainerStarted","Data":"c2af6e5f44037182ee877358797a830aaac134a18193a8b002e7e914ba25012e"}
Sep 09 13:03:15.122001 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:15.121818    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:15.145205 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:15.145145    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" podStartSLOduration=2.145127044 podStartE2EDuration="2.145127044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:15.144101293 +0000 UTC m=+881.144767393" watchObservedRunningTime="2026-09-09 13:03:15.145127044 +0000 UTC m=+881.145793146"
Sep 09 13:03:15.145737 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:15.145703    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:15.151760 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:15.151736    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:16.125703 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:16.125671    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:16.126495 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:16.126479    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:18.410379 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.410346    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"]
Sep 09 13:03:18.412656 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.412620    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:18.415159 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.415138    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d0fb1d64-dockercfg-qf9sj\""
Sep 09 13:03:18.415364 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.415350    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d0fb1d64\""
Sep 09 13:03:18.423381 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.423361    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"]
Sep 09 13:03:18.581707 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.581664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2dqk\" (UniqueName: \"kubernetes.io/projected/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-kube-api-access-x2dqk\") pod \"payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht\" (UID: \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\") " pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:18.581867 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.581846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-plugins-config-volume\") pod \"payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht\" (UID: \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\") " pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:18.682689 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.682437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-plugins-config-volume\") pod \"payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht\" (UID: \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\") " pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:18.684181 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.684161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2dqk\" (UniqueName: \"kubernetes.io/projected/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-kube-api-access-x2dqk\") pod \"payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht\" (UID: \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\") " pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:18.684707 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.684072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-plugins-config-volume\") pod \"payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht\" (UID: \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\") " pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:18.693089 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.693068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2dqk\" (UniqueName: \"kubernetes.io/projected/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-kube-api-access-x2dqk\") pod \"payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht\" (UID: \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\") " pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:18.749808 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:18.749784    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:19.045572 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:19.045545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"]
Sep 09 13:03:19.047428 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:19.047401    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c5f361_ab5f_4799_b80a_fb2d7e28c05f.slice/crio-06f53f8e023f89238aa25f61c94368283b5f29b8c74547885638f15dc4ce8c7c WatchSource:0}: Error finding container 06f53f8e023f89238aa25f61c94368283b5f29b8c74547885638f15dc4ce8c7c: Status 404 returned error can't find the container with id 06f53f8e023f89238aa25f61c94368283b5f29b8c74547885638f15dc4ce8c7c
Sep 09 13:03:19.144282 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:19.144252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht" event={"ID":"80c5f361-ab5f-4799-b80a-fb2d7e28c05f","Type":"ContainerStarted","Data":"db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e"}
Sep 09 13:03:19.144443 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:19.144410    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:19.144492 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:19.144451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht" event={"ID":"80c5f361-ab5f-4799-b80a-fb2d7e28c05f","Type":"ContainerStarted","Data":"06f53f8e023f89238aa25f61c94368283b5f29b8c74547885638f15dc4ce8c7c"}
Sep 09 13:03:19.161612 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:19.161567    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht" podStartSLOduration=1.161551985 podStartE2EDuration="1.161551985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:19.159373325 +0000 UTC m=+885.160039430" watchObservedRunningTime="2026-09-09 13:03:19.161551985 +0000 UTC m=+885.162218084"
Sep 09 13:03:21.131416 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:21.131381    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:03:23.111189 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:23.111156    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:03:24.117305 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:24.117277    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:03:27.190042 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:27.189999    2577 generic.go:356] "Generic (PLEG): container finished" podID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerID="b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa" exitCode=0
Sep 09 13:03:27.190570 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:27.190114    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" event={"ID":"cbcbefac-4382-4a7a-93ea-9e5492576609","Type":"ContainerDied","Data":"b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa"}
Sep 09 13:03:27.190662 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:27.190576    2577 scope.go:117] "RemoveContainer" containerID="b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa"
Sep 09 13:03:27.192046 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:27.191987    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:27.192164 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:27.192080    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:27.192164 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:27.192131    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:28.197474 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:28.197427    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" event={"ID":"cbcbefac-4382-4a7a-93ea-9e5492576609","Type":"ContainerStarted","Data":"4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c"}
Sep 09 13:03:29.024961 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.024918    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"]
Sep 09 13:03:29.027318 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.027302    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:29.029583 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.029564    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-b4ea80\""
Sep 09 13:03:29.029818 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.029589    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-b4ea80-dockercfg-c4stf\""
Sep 09 13:03:29.036122 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.036084    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"]
Sep 09 13:03:29.079140 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.079103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdb3b7ef-972f-4ade-9987-c9849c787ed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx\" (UID: \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:29.079494 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.079474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-952dw\" (UniqueName: \"kubernetes.io/projected/cdb3b7ef-972f-4ade-9987-c9849c787ed0-kube-api-access-952dw\") pod \"payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx\" (UID: \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:29.165807 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.165780    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:29.166030 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.165952    2577 patch_prober.go:28] interesting pod/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Sep 09 13:03:29.166142 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.166124    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Sep 09 13:03:29.186661 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.184170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-952dw\" (UniqueName: \"kubernetes.io/projected/cdb3b7ef-972f-4ade-9987-c9849c787ed0-kube-api-access-952dw\") pod \"payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx\" (UID: \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:29.186661 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.184369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdb3b7ef-972f-4ade-9987-c9849c787ed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx\" (UID: \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:29.186661 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.185608    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdb3b7ef-972f-4ade-9987-c9849c787ed0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx\" (UID: \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:29.198674 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.198622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-952dw\" (UniqueName: \"kubernetes.io/projected/cdb3b7ef-972f-4ade-9987-c9849c787ed0-kube-api-access-952dw\") pod \"payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx\" (UID: \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:29.339457 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.339363    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:29.663982 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:29.663924    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"]
Sep 09 13:03:29.666696 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:29.666650    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb3b7ef_972f_4ade_9987_c9849c787ed0.slice/crio-c29be6e0e4adb866c6212f43ce0702f87b02c496af9aa25863f3a632b31b96ae WatchSource:0}: Error finding container c29be6e0e4adb866c6212f43ce0702f87b02c496af9aa25863f3a632b31b96ae: Status 404 returned error can't find the container with id c29be6e0e4adb866c6212f43ce0702f87b02c496af9aa25863f3a632b31b96ae
Sep 09 13:03:30.151546 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:30.151517    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:30.166146 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:30.166121    2577 patch_prober.go:28] interesting pod/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Sep 09 13:03:30.166281 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:30.166166    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Sep 09 13:03:30.254353 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:30.253838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx" event={"ID":"cdb3b7ef-972f-4ade-9987-c9849c787ed0","Type":"ContainerStarted","Data":"9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd"}
Sep 09 13:03:30.254353 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:30.253943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx" event={"ID":"cdb3b7ef-972f-4ade-9987-c9849c787ed0","Type":"ContainerStarted","Data":"c29be6e0e4adb866c6212f43ce0702f87b02c496af9aa25863f3a632b31b96ae"}
Sep 09 13:03:30.254353 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:30.254000    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:30.277367 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:30.277313    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx" podStartSLOduration=1.277279288 podStartE2EDuration="1.277279288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:30.275411885 +0000 UTC m=+896.276077988" watchObservedRunningTime="2026-09-09 13:03:30.277279288 +0000 UTC m=+896.277945388"
Sep 09 13:03:31.166512 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:31.166474    2577 patch_prober.go:28] interesting pod/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Sep 09 13:03:31.166724 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:31.166549    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Sep 09 13:03:32.165772 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:32.165733    2577 patch_prober.go:28] interesting pod/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Sep 09 13:03:32.166164 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:32.165791    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Sep 09 13:03:32.167880 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:32.167771    2577 patch_prober.go:28] interesting pod/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 13:03:32.167880 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:32.167820    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 13:03:33.165792 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:33.165759    2577 patch_prober.go:28] interesting pod/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body=
Sep 09 13:03:33.166228 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:33.165817    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused"
Sep 09 13:03:34.169243 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.169217    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:34.169622 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.169585    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:34.170347 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.170324    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:03:34.276214 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.276149    2577 generic.go:356] "Generic (PLEG): container finished" podID="dc310562-84e3-40c6-8aa0-b410feccd443" containerID="1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473" exitCode=0
Sep 09 13:03:34.276339 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.276217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" event={"ID":"dc310562-84e3-40c6-8aa0-b410feccd443","Type":"ContainerDied","Data":"1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473"}
Sep 09 13:03:34.276706 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.276692    2577 scope.go:117] "RemoveContainer" containerID="1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473"
Sep 09 13:03:34.277701 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.277672    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:34.277774 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.277742    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:34.277809 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:34.277779    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:03:35.286488 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:35.286444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" event={"ID":"dc310562-84e3-40c6-8aa0-b410feccd443","Type":"ContainerStarted","Data":"3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec"}
Sep 09 13:03:36.145699 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:36.145667    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:36.145897 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:36.145812    2577 patch_prober.go:28] interesting pod/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 13:03:36.145897 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:36.145861    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 13:03:37.145833 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:37.145799    2577 patch_prober.go:28] interesting pod/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 13:03:37.146227 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:37.145857    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 13:03:38.145990 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:38.145954    2577 patch_prober.go:28] interesting pod/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 13:03:38.146379 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:38.146021    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 13:03:39.145522 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:39.145484    2577 patch_prober.go:28] interesting pod/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 13:03:39.145731 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:39.145543    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 13:03:40.145088 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:40.145055    2577 patch_prober.go:28] interesting pod/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Sep 09 13:03:40.145486 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:40.145112    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Sep 09 13:03:41.149037 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:41.149003    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:41.149480 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:41.149297    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:41.149901 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:41.149887    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:41.261598 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:41.261569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:03:42.609086 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.609054    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"]
Sep 09 13:03:42.611912 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.611894    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:42.614325 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.614299    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-7c991b-dockercfg-lj7hr\""
Sep 09 13:03:42.614325 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.614321    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-7c991b\""
Sep 09 13:03:42.622767 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.622742    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"]
Sep 09 13:03:42.726916 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.726880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqzdn\" (UniqueName: \"kubernetes.io/projected/dd066d77-17a4-4bdb-b495-316ae86d9a1b-kube-api-access-bqzdn\") pod \"payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk\" (UID: \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:42.727111 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.727022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd066d77-17a4-4bdb-b495-316ae86d9a1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk\" (UID: \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:42.828237 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.828195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqzdn\" (UniqueName: \"kubernetes.io/projected/dd066d77-17a4-4bdb-b495-316ae86d9a1b-kube-api-access-bqzdn\") pod \"payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk\" (UID: \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:42.828417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.828277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd066d77-17a4-4bdb-b495-316ae86d9a1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk\" (UID: \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:42.829007 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.828989    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd066d77-17a4-4bdb-b495-316ae86d9a1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk\" (UID: \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:42.836535 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.836513    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqzdn\" (UniqueName: \"kubernetes.io/projected/dd066d77-17a4-4bdb-b495-316ae86d9a1b-kube-api-access-bqzdn\") pod \"payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk\" (UID: \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:42.922116 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:42.922076    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:43.196556 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:43.196531    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"]
Sep 09 13:03:43.197584 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:43.197563    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd066d77_17a4_4bdb_b495_316ae86d9a1b.slice/crio-d48add5b04ad841129421c3cc6cc094313b6cc15f9b6b150676b8aa62da43617 WatchSource:0}: Error finding container d48add5b04ad841129421c3cc6cc094313b6cc15f9b6b150676b8aa62da43617: Status 404 returned error can't find the container with id d48add5b04ad841129421c3cc6cc094313b6cc15f9b6b150676b8aa62da43617
Sep 09 13:03:43.335827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:43.335781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk" event={"ID":"dd066d77-17a4-4bdb-b495-316ae86d9a1b","Type":"ContainerStarted","Data":"74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8"}
Sep 09 13:03:43.335827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:43.335828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk" event={"ID":"dd066d77-17a4-4bdb-b495-316ae86d9a1b","Type":"ContainerStarted","Data":"d48add5b04ad841129421c3cc6cc094313b6cc15f9b6b150676b8aa62da43617"}
Sep 09 13:03:43.336038 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:43.335920    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:43.352360 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:43.352315    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk" podStartSLOduration=1.3522997860000001 podStartE2EDuration="1.352299786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:43.351392552 +0000 UTC m=+909.352058655" watchObservedRunningTime="2026-09-09 13:03:43.352299786 +0000 UTC m=+909.352965929"
Sep 09 13:03:48.472521 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.472488    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"]
Sep 09 13:03:48.472990 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.472832    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht" podUID="80c5f361-ab5f-4799-b80a-fb2d7e28c05f" containerName="payload-processing" containerID="cri-o://db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e" gracePeriod=30
Sep 09 13:03:48.716694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.716626    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:48.879250 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.879218    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-plugins-config-volume\") pod \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\" (UID: \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\") "
Sep 09 13:03:48.879399 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.879289    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2dqk\" (UniqueName: \"kubernetes.io/projected/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-kube-api-access-x2dqk\") pod \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\" (UID: \"80c5f361-ab5f-4799-b80a-fb2d7e28c05f\") "
Sep 09 13:03:48.879709 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.879678    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80c5f361-ab5f-4799-b80a-fb2d7e28c05f" (UID: "80c5f361-ab5f-4799-b80a-fb2d7e28c05f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:03:48.881623 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.881598    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-kube-api-access-x2dqk" (OuterVolumeSpecName: "kube-api-access-x2dqk") pod "80c5f361-ab5f-4799-b80a-fb2d7e28c05f" (UID: "80c5f361-ab5f-4799-b80a-fb2d7e28c05f"). InnerVolumeSpecName "kube-api-access-x2dqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:03:48.924443 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.924410    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"]
Sep 09 13:03:48.924755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.924701    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy" containerID="cri-o://3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec" gracePeriod=30
Sep 09 13:03:48.980586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.980560    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:48.980586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:48.980583    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2dqk\" (UniqueName: \"kubernetes.io/projected/80c5f361-ab5f-4799-b80a-fb2d7e28c05f-kube-api-access-x2dqk\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:49.374248 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.374162    2577 generic.go:356] "Generic (PLEG): container finished" podID="80c5f361-ab5f-4799-b80a-fb2d7e28c05f" containerID="db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e" exitCode=0
Sep 09 13:03:49.374248 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.374206    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"
Sep 09 13:03:49.374411 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.374245    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht" event={"ID":"80c5f361-ab5f-4799-b80a-fb2d7e28c05f","Type":"ContainerDied","Data":"db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e"}
Sep 09 13:03:49.374411 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.374315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht" event={"ID":"80c5f361-ab5f-4799-b80a-fb2d7e28c05f","Type":"ContainerDied","Data":"06f53f8e023f89238aa25f61c94368283b5f29b8c74547885638f15dc4ce8c7c"}
Sep 09 13:03:49.374411 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.374348    2577 scope.go:117] "RemoveContainer" containerID="db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e"
Sep 09 13:03:49.383808 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.383792    2577 scope.go:117] "RemoveContainer" containerID="db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e"
Sep 09 13:03:49.384079 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:03:49.384059    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e\": container with ID starting with db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e not found: ID does not exist" containerID="db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e"
Sep 09 13:03:49.384143 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.384091    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e"} err="failed to get container status \"db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e\": rpc error: code = NotFound desc = could not find container \"db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e\": container with ID starting with db7f07cf6c738f325379304ca390035444180c246fbeb27528ff34a96cfd518e not found: ID does not exist"
Sep 09 13:03:49.397725 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.397704    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"]
Sep 09 13:03:49.406125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:49.406105    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d0fb1d64-6bb5db5cc-p9bht"]
Sep 09 13:03:50.515253 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:50.515219    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80c5f361-ab5f-4799-b80a-fb2d7e28c05f" path="/var/lib/kubelet/pods/80c5f361-ab5f-4799-b80a-fb2d7e28c05f/volumes"
Sep 09 13:03:52.459851 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.459818    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"]
Sep 09 13:03:52.460339 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.460319    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80c5f361-ab5f-4799-b80a-fb2d7e28c05f" containerName="payload-processing"
Sep 09 13:03:52.460404 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.460343    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="80c5f361-ab5f-4799-b80a-fb2d7e28c05f" containerName="payload-processing"
Sep 09 13:03:52.460477 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.460464    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="80c5f361-ab5f-4799-b80a-fb2d7e28c05f" containerName="payload-processing"
Sep 09 13:03:52.463850 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.463823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.466135 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.466116    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-776d31f2-serving-cert\""
Sep 09 13:03:52.475520 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.475497    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"]
Sep 09 13:03:52.610931 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.610895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqdqz\" (UniqueName: \"kubernetes.io/projected/9fb27dfe-ad0d-4e25-ac34-170599b61535-kube-api-access-dqdqz\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.611119 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.610960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-metrics-certs\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.611226 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.611200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-maas-api-tls\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.611312 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.611272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb27dfe-ad0d-4e25-ac34-170599b61535-openshift-service-ca\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.712654 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.712581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-maas-api-tls\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.712654 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.712616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb27dfe-ad0d-4e25-ac34-170599b61535-openshift-service-ca\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.712802 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.712754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqdqz\" (UniqueName: \"kubernetes.io/projected/9fb27dfe-ad0d-4e25-ac34-170599b61535-kube-api-access-dqdqz\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.712802 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.712794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-metrics-certs\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.713270 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.713247    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb27dfe-ad0d-4e25-ac34-170599b61535-openshift-service-ca\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.715218 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.715194    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-metrics-certs\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.715311 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.715260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-maas-api-tls\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.721008 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.720988    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqdqz\" (UniqueName: \"kubernetes.io/projected/9fb27dfe-ad0d-4e25-ac34-170599b61535-kube-api-access-dqdqz\") pod \"maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:52.775947 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:52.775918    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:53.050942 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.049261    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"]
Sep 09 13:03:53.161096 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.161065    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"]
Sep 09 13:03:53.163832 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.163811    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:53.166230 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.166189    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-776d31f2\""
Sep 09 13:03:53.166486 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.166466    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-776d31f2-dockercfg-2r49p\""
Sep 09 13:03:53.179522 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.179496    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"]
Sep 09 13:03:53.192491 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.192466    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"]
Sep 09 13:03:53.194652 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.194620    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:53.206445 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.206421    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"]
Sep 09 13:03:53.318333 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.318255    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44\" (UID: \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:53.318333 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.318309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-plugins-config-volume\") pod \"payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl\" (UID: \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\") " pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:53.318569 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.318410    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5chwt\" (UniqueName: \"kubernetes.io/projected/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-kube-api-access-5chwt\") pod \"payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl\" (UID: \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\") " pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:53.318569 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.318446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8wwr\" (UniqueName: \"kubernetes.io/projected/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-kube-api-access-m8wwr\") pod \"payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44\" (UID: \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:53.396646 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.396596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264" event={"ID":"9fb27dfe-ad0d-4e25-ac34-170599b61535","Type":"ContainerStarted","Data":"6a1e99070dd9ec8068b9e4b1316484eec037f86256add0f0dac30016c9f2e561"}
Sep 09 13:03:53.419730 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.419689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5chwt\" (UniqueName: \"kubernetes.io/projected/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-kube-api-access-5chwt\") pod \"payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl\" (UID: \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\") " pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:53.419730 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.419730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8wwr\" (UniqueName: \"kubernetes.io/projected/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-kube-api-access-m8wwr\") pod \"payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44\" (UID: \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:53.419947 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.419825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44\" (UID: \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:53.419947 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.419899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-plugins-config-volume\") pod \"payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl\" (UID: \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\") " pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:53.420539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.420517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44\" (UID: \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:53.420682 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.420598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-plugins-config-volume\") pod \"payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl\" (UID: \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\") " pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:53.427750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.427728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8wwr\" (UniqueName: \"kubernetes.io/projected/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-kube-api-access-m8wwr\") pod \"payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44\" (UID: \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:53.427840 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.427757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5chwt\" (UniqueName: \"kubernetes.io/projected/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-kube-api-access-5chwt\") pod \"payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl\" (UID: \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\") " pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:53.473513 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.473490    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:53.504224 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.504200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:53.751064 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.751040    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"]
Sep 09 13:03:53.752724 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:53.752701    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae8b19e_ceff_4137_9d06_3f4f2bdc2747.slice/crio-bfd736d5d77974676304d93ac2e5ed159ae4046cba5fcf5ed984172f9570f027 WatchSource:0}: Error finding container bfd736d5d77974676304d93ac2e5ed159ae4046cba5fcf5ed984172f9570f027: Status 404 returned error can't find the container with id bfd736d5d77974676304d93ac2e5ed159ae4046cba5fcf5ed984172f9570f027
Sep 09 13:03:53.781227 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:53.781206    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"]
Sep 09 13:03:53.784698 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:03:53.784663    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43efd7b_72c0_41e8_8048_0eb98cb4cd58.slice/crio-87bd8d30e43b97714bbe8bd2193dcd7478588bb382300195ec6a96fbca7004ce WatchSource:0}: Error finding container 87bd8d30e43b97714bbe8bd2193dcd7478588bb382300195ec6a96fbca7004ce: Status 404 returned error can't find the container with id 87bd8d30e43b97714bbe8bd2193dcd7478588bb382300195ec6a96fbca7004ce
Sep 09 13:03:54.160649 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.160612    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:54.327816 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.327736    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64hr4\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-kube-api-access-64hr4\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.327816 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.327775    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-certs\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.327816 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.327806    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-socket\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.328066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.327841    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-data\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.328066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.327860    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-envoy\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.328066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.327915    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-istio-token\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.328066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.327958    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc310562-84e3-40c6-8aa0-b410feccd443-istiod-ca-cert\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.328066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.327983    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-credential-socket\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.328066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.328016    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc310562-84e3-40c6-8aa0-b410feccd443-istio-podinfo\") pod \"dc310562-84e3-40c6-8aa0-b410feccd443\" (UID: \"dc310562-84e3-40c6-8aa0-b410feccd443\") "
Sep 09 13:03:54.328362 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.328112    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:03:54.328362 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.328293    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:03:54.328362 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.328320    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.328514 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.328414    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:03:54.328514 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.328416    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc310562-84e3-40c6-8aa0-b410feccd443-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:03:54.328681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.328617    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:03:54.330298 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.330274    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:03:54.330418 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.330310    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-kube-api-access-64hr4" (OuterVolumeSpecName: "kube-api-access-64hr4") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "kube-api-access-64hr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:03:54.330418 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.330363    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:03:54.330677 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.330658    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc310562-84e3-40c6-8aa0-b410feccd443-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc310562-84e3-40c6-8aa0-b410feccd443" (UID: "dc310562-84e3-40c6-8aa0-b410feccd443"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:03:54.342801 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.342784    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:03:54.404873 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.404840    2577 generic.go:356] "Generic (PLEG): container finished" podID="dc310562-84e3-40c6-8aa0-b410feccd443" containerID="3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec" exitCode=0
Sep 09 13:03:54.405020 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.404895    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"
Sep 09 13:03:54.405020 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.404925    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" event={"ID":"dc310562-84e3-40c6-8aa0-b410feccd443","Type":"ContainerDied","Data":"3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec"}
Sep 09 13:03:54.405020 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.404987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq" event={"ID":"dc310562-84e3-40c6-8aa0-b410feccd443","Type":"ContainerDied","Data":"c2af6e5f44037182ee877358797a830aaac134a18193a8b002e7e914ba25012e"}
Sep 09 13:03:54.405020 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.405016    2577 scope.go:117] "RemoveContainer" containerID="3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec"
Sep 09 13:03:54.406778 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.406596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl" event={"ID":"a43efd7b-72c0-41e8-8048-0eb98cb4cd58","Type":"ContainerStarted","Data":"b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308"}
Sep 09 13:03:54.406778 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.406666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl" event={"ID":"a43efd7b-72c0-41e8-8048-0eb98cb4cd58","Type":"ContainerStarted","Data":"87bd8d30e43b97714bbe8bd2193dcd7478588bb382300195ec6a96fbca7004ce"}
Sep 09 13:03:54.407192 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.407171    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:03:54.409240 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.409213    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44" event={"ID":"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747","Type":"ContainerStarted","Data":"f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a"}
Sep 09 13:03:54.409373 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.409262    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44" event={"ID":"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747","Type":"ContainerStarted","Data":"bfd736d5d77974676304d93ac2e5ed159ae4046cba5fcf5ed984172f9570f027"}
Sep 09 13:03:54.409373 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.409304    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:03:54.418823 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.416485    2577 scope.go:117] "RemoveContainer" containerID="1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473"
Sep 09 13:03:54.427624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.427582    2577 scope.go:117] "RemoveContainer" containerID="3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec"
Sep 09 13:03:54.427927 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:03:54.427900    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec\": container with ID starting with 3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec not found: ID does not exist" containerID="3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec"
Sep 09 13:03:54.428022 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.427936    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec"} err="failed to get container status \"3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec\": rpc error: code = NotFound desc = could not find container \"3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec\": container with ID starting with 3c410c51855f95729b9b6e483a908587d291dec311b1d18da7c2aff65f34f9ec not found: ID does not exist"
Sep 09 13:03:54.428022 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.427959    2577 scope.go:117] "RemoveContainer" containerID="1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473"
Sep 09 13:03:54.428289 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:03:54.428233    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473\": container with ID starting with 1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473 not found: ID does not exist" containerID="1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473"
Sep 09 13:03:54.428378 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.428297    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473"} err="failed to get container status \"1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473\": rpc error: code = NotFound desc = could not find container \"1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473\": container with ID starting with 1d583ca377d416fb56d4a184de5f69f47419a75a9c51ebb7e659f19740ad0473 not found: ID does not exist"
Sep 09 13:03:54.429578 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.429396    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64hr4\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-kube-api-access-64hr4\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.429578 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.429421    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.429578 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.429434    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.429578 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.429515    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.429578 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.429528    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc310562-84e3-40c6-8aa0-b410feccd443-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.429578 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.429541    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc310562-84e3-40c6-8aa0-b410feccd443-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.429578 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.429549    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc310562-84e3-40c6-8aa0-b410feccd443-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.429578 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.429557    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc310562-84e3-40c6-8aa0-b410feccd443-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:03:54.430409 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.430374    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl" podStartSLOduration=1.4303623509999999 podStartE2EDuration="1.430362351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:54.429236741 +0000 UTC m=+920.429902842" watchObservedRunningTime="2026-09-09 13:03:54.430362351 +0000 UTC m=+920.431028466"
Sep 09 13:03:54.443078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.443055    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"]
Sep 09 13:03:54.448537 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.448516    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d0fb1d64-openshift-default-6774c9b7c8-2dwtq"]
Sep 09 13:03:54.461608 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.461572    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44" podStartSLOduration=1.461560801 podStartE2EDuration="1.461560801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:54.460890331 +0000 UTC m=+920.461556435" watchObservedRunningTime="2026-09-09 13:03:54.461560801 +0000 UTC m=+920.462226959"
Sep 09 13:03:54.523194 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:54.523170    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" path="/var/lib/kubelet/pods/dc310562-84e3-40c6-8aa0-b410feccd443/volumes"
Sep 09 13:03:57.426728 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:57.426694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264" event={"ID":"9fb27dfe-ad0d-4e25-ac34-170599b61535","Type":"ContainerStarted","Data":"ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e"}
Sep 09 13:03:57.427108 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:57.426827    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:03:57.445524 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:03:57.445442    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264" podStartSLOduration=1.622397568 podStartE2EDuration="5.445428606s" totalImagesPullingTime="3.823031038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:57.444760783 +0000 UTC m=+923.445426884" watchObservedRunningTime="2026-09-09 13:03:57.445428606 +0000 UTC m=+923.446094705"
Sep 09 13:04:03.436973 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:03.436944    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:04:05.416771 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:05.416742    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:04:06.421311 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:06.421283    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:04:23.714341 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:23.714308    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"]
Sep 09 13:04:23.714884 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:23.714607    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264" podUID="9fb27dfe-ad0d-4e25-ac34-170599b61535" containerName="maas-api" containerID="cri-o://ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e" gracePeriod=30
Sep 09 13:04:23.769621 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:23.769590    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"]
Sep 09 13:04:23.769835 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:23.769809    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl" podUID="a43efd7b-72c0-41e8-8048-0eb98cb4cd58" containerName="payload-processing" containerID="cri-o://b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308" gracePeriod=30
Sep 09 13:04:23.778110 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:23.778086    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"]
Sep 09 13:04:23.778588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:23.778322    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44" podUID="2ae8b19e-ceff-4137-9d06-3f4f2bdc2747" containerName="payload-pre-processing" containerID="cri-o://f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a" gracePeriod=30
Sep 09 13:04:24.057264 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.056993    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:04:24.076238 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.076085    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:04:24.100816 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.100792    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:04:24.179365 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179335    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqdqz\" (UniqueName: \"kubernetes.io/projected/9fb27dfe-ad0d-4e25-ac34-170599b61535-kube-api-access-dqdqz\") pod \"9fb27dfe-ad0d-4e25-ac34-170599b61535\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") "
Sep 09 13:04:24.179365 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179367    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-metrics-certs\") pod \"9fb27dfe-ad0d-4e25-ac34-170599b61535\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") "
Sep 09 13:04:24.179571 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179434    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb27dfe-ad0d-4e25-ac34-170599b61535-openshift-service-ca\") pod \"9fb27dfe-ad0d-4e25-ac34-170599b61535\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") "
Sep 09 13:04:24.179571 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179472    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-plugins-config-volume\") pod \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\" (UID: \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\") "
Sep 09 13:04:24.179571 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179507    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5chwt\" (UniqueName: \"kubernetes.io/projected/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-kube-api-access-5chwt\") pod \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\" (UID: \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\") "
Sep 09 13:04:24.179571 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179534    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-maas-api-tls\") pod \"9fb27dfe-ad0d-4e25-ac34-170599b61535\" (UID: \"9fb27dfe-ad0d-4e25-ac34-170599b61535\") "
Sep 09 13:04:24.179571 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-plugins-config-volume\") pod \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\" (UID: \"a43efd7b-72c0-41e8-8048-0eb98cb4cd58\") "
Sep 09 13:04:24.179890 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179784    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8wwr\" (UniqueName: \"kubernetes.io/projected/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-kube-api-access-m8wwr\") pod \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\" (UID: \"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747\") "
Sep 09 13:04:24.179973 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.179861    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb27dfe-ad0d-4e25-ac34-170599b61535-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9fb27dfe-ad0d-4e25-ac34-170599b61535" (UID: "9fb27dfe-ad0d-4e25-ac34-170599b61535"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:04:24.180110 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.180081    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a43efd7b-72c0-41e8-8048-0eb98cb4cd58" (UID: "a43efd7b-72c0-41e8-8048-0eb98cb4cd58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:04:24.180166 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.180141    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb27dfe-ad0d-4e25-ac34-170599b61535-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:24.180745 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.180711    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2ae8b19e-ceff-4137-9d06-3f4f2bdc2747" (UID: "2ae8b19e-ceff-4137-9d06-3f4f2bdc2747"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:04:24.181924 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.181900    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-kube-api-access-5chwt" (OuterVolumeSpecName: "kube-api-access-5chwt") pod "a43efd7b-72c0-41e8-8048-0eb98cb4cd58" (UID: "a43efd7b-72c0-41e8-8048-0eb98cb4cd58"). InnerVolumeSpecName "kube-api-access-5chwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:04:24.182021 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.181993    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb27dfe-ad0d-4e25-ac34-170599b61535-kube-api-access-dqdqz" (OuterVolumeSpecName: "kube-api-access-dqdqz") pod "9fb27dfe-ad0d-4e25-ac34-170599b61535" (UID: "9fb27dfe-ad0d-4e25-ac34-170599b61535"). InnerVolumeSpecName "kube-api-access-dqdqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:04:24.182073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.182055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-kube-api-access-m8wwr" (OuterVolumeSpecName: "kube-api-access-m8wwr") pod "2ae8b19e-ceff-4137-9d06-3f4f2bdc2747" (UID: "2ae8b19e-ceff-4137-9d06-3f4f2bdc2747"). InnerVolumeSpecName "kube-api-access-m8wwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:04:24.182197 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.182182    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9fb27dfe-ad0d-4e25-ac34-170599b61535" (UID: "9fb27dfe-ad0d-4e25-ac34-170599b61535"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:04:24.182239 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.182196    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9fb27dfe-ad0d-4e25-ac34-170599b61535" (UID: "9fb27dfe-ad0d-4e25-ac34-170599b61535"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:04:24.280580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.280496    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqdqz\" (UniqueName: \"kubernetes.io/projected/9fb27dfe-ad0d-4e25-ac34-170599b61535-kube-api-access-dqdqz\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:24.280580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.280526    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:24.280580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.280539    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:24.280580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.280551    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5chwt\" (UniqueName: \"kubernetes.io/projected/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-kube-api-access-5chwt\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:24.280580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.280562    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9fb27dfe-ad0d-4e25-ac34-170599b61535-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:24.280580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.280574    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a43efd7b-72c0-41e8-8048-0eb98cb4cd58-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:24.280890 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.280585    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8wwr\" (UniqueName: \"kubernetes.io/projected/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747-kube-api-access-m8wwr\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:24.559681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.559591    2577 generic.go:356] "Generic (PLEG): container finished" podID="2ae8b19e-ceff-4137-9d06-3f4f2bdc2747" containerID="f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a" exitCode=0
Sep 09 13:04:24.559681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.559655    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"
Sep 09 13:04:24.559805 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.559679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44" event={"ID":"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747","Type":"ContainerDied","Data":"f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a"}
Sep 09 13:04:24.559805 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.559729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44" event={"ID":"2ae8b19e-ceff-4137-9d06-3f4f2bdc2747","Type":"ContainerDied","Data":"bfd736d5d77974676304d93ac2e5ed159ae4046cba5fcf5ed984172f9570f027"}
Sep 09 13:04:24.559805 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.559757    2577 scope.go:117] "RemoveContainer" containerID="f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a"
Sep 09 13:04:24.561236 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.561211    2577 generic.go:356] "Generic (PLEG): container finished" podID="9fb27dfe-ad0d-4e25-ac34-170599b61535" containerID="ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e" exitCode=0
Sep 09 13:04:24.561352 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.561274    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"
Sep 09 13:04:24.561352 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.561297    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264" event={"ID":"9fb27dfe-ad0d-4e25-ac34-170599b61535","Type":"ContainerDied","Data":"ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e"}
Sep 09 13:04:24.561352 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.561336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264" event={"ID":"9fb27dfe-ad0d-4e25-ac34-170599b61535","Type":"ContainerDied","Data":"6a1e99070dd9ec8068b9e4b1316484eec037f86256add0f0dac30016c9f2e561"}
Sep 09 13:04:24.563101 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.563078    2577 generic.go:356] "Generic (PLEG): container finished" podID="a43efd7b-72c0-41e8-8048-0eb98cb4cd58" containerID="b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308" exitCode=0
Sep 09 13:04:24.563223 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.563115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl" event={"ID":"a43efd7b-72c0-41e8-8048-0eb98cb4cd58","Type":"ContainerDied","Data":"b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308"}
Sep 09 13:04:24.563223 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.563127    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"
Sep 09 13:04:24.563223 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.563153    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl" event={"ID":"a43efd7b-72c0-41e8-8048-0eb98cb4cd58","Type":"ContainerDied","Data":"87bd8d30e43b97714bbe8bd2193dcd7478588bb382300195ec6a96fbca7004ce"}
Sep 09 13:04:24.568883 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.568866    2577 scope.go:117] "RemoveContainer" containerID="f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a"
Sep 09 13:04:24.569161 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:04:24.569141    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a\": container with ID starting with f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a not found: ID does not exist" containerID="f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a"
Sep 09 13:04:24.569222 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.569168    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a"} err="failed to get container status \"f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a\": rpc error: code = NotFound desc = could not find container \"f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a\": container with ID starting with f0a7a8fefb417cc26cd484e264c94ba9a125c511cda727dde56884e89867860a not found: ID does not exist"
Sep 09 13:04:24.569222 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.569184    2577 scope.go:117] "RemoveContainer" containerID="ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e"
Sep 09 13:04:24.577092 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.577075    2577 scope.go:117] "RemoveContainer" containerID="ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e"
Sep 09 13:04:24.577357 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:04:24.577340    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e\": container with ID starting with ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e not found: ID does not exist" containerID="ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e"
Sep 09 13:04:24.577428 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.577383    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e"} err="failed to get container status \"ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e\": rpc error: code = NotFound desc = could not find container \"ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e\": container with ID starting with ff0b6888586aa660ba5afc92ff5f974f3be8f9e5da3fbfdd54feddbf58f4b32e not found: ID does not exist"
Sep 09 13:04:24.577428 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.577398    2577 scope.go:117] "RemoveContainer" containerID="b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308"
Sep 09 13:04:24.583400 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.583355    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"]
Sep 09 13:04:24.585135 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.585104    2577 scope.go:117] "RemoveContainer" containerID="b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308"
Sep 09 13:04:24.585365 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:04:24.585347    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308\": container with ID starting with b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308 not found: ID does not exist" containerID="b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308"
Sep 09 13:04:24.585422 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.585372    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308"} err="failed to get container status \"b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308\": rpc error: code = NotFound desc = could not find container \"b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308\": container with ID starting with b050f1c81938141f5c1277bb19dc9b9afcecb86af61d481a8de4a11ddc0bd308 not found: ID does not exist"
Sep 09 13:04:24.586344 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.586311    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-776d31f2-8545d7bf86-zhxtl"]
Sep 09 13:04:24.596516 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.596495    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"]
Sep 09 13:04:24.600266 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.600237    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-776d31f2-59cd964df8-krd44"]
Sep 09 13:04:24.614408 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.614390    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"]
Sep 09 13:04:24.618375 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.618355    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-776d31f2-5ccbdf498-wg264"]
Sep 09 13:04:24.884371 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884341    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"]
Sep 09 13:04:24.884879 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884859    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ae8b19e-ceff-4137-9d06-3f4f2bdc2747" containerName="payload-pre-processing"
Sep 09 13:04:24.884959 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884882    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ae8b19e-ceff-4137-9d06-3f4f2bdc2747" containerName="payload-pre-processing"
Sep 09 13:04:24.884959 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884903    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb27dfe-ad0d-4e25-ac34-170599b61535" containerName="maas-api"
Sep 09 13:04:24.884959 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884913    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb27dfe-ad0d-4e25-ac34-170599b61535" containerName="maas-api"
Sep 09 13:04:24.884959 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884925    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy"
Sep 09 13:04:24.884959 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884934    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy"
Sep 09 13:04:24.884959 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884946    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy"
Sep 09 13:04:24.884959 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884953    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy"
Sep 09 13:04:24.885274 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884962    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a43efd7b-72c0-41e8-8048-0eb98cb4cd58" containerName="payload-processing"
Sep 09 13:04:24.885274 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.884970    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a43efd7b-72c0-41e8-8048-0eb98cb4cd58" containerName="payload-processing"
Sep 09 13:04:24.885274 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.885052    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a43efd7b-72c0-41e8-8048-0eb98cb4cd58" containerName="payload-processing"
Sep 09 13:04:24.885274 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.885063    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fb27dfe-ad0d-4e25-ac34-170599b61535" containerName="maas-api"
Sep 09 13:04:24.885274 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.885072    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ae8b19e-ceff-4137-9d06-3f4f2bdc2747" containerName="payload-pre-processing"
Sep 09 13:04:24.885274 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.885082    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy"
Sep 09 13:04:24.885274 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.885090    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc310562-84e3-40c6-8aa0-b410feccd443" containerName="istio-proxy"
Sep 09 13:04:24.889526 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.889506    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.891802 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.891783    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0175d9d3-openshift-default-dockercfg-vwgjx\""
Sep 09 13:04:24.897958 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.897936    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"]
Sep 09 13:04:24.987595 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-envoy\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.987794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-certs\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.987794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-istio-token\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.987794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58136754-4a10-4398-bd0b-e869b8a36b69-istiod-ca-cert\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.987963 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987824    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-credential-socket\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.987963 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987858    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-data\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.987963 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58136754-4a10-4398-bd0b-e869b8a36b69-istio-podinfo\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.987963 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-socket\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:24.988156 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:24.987970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb6qq\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-kube-api-access-xb6qq\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089150 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089112    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-envoy\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089338 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-certs\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089338 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-istio-token\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089338 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58136754-4a10-4398-bd0b-e869b8a36b69-istiod-ca-cert\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089338 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-credential-socket\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089338 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-data\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089606 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58136754-4a10-4398-bd0b-e869b8a36b69-istio-podinfo\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089606 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-socket\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089606 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb6qq\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-kube-api-access-xb6qq\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089796 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-certs\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089796 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089711    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-credential-socket\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089899 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-data\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.089950 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.089936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-socket\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.090314 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.090288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58136754-4a10-4398-bd0b-e869b8a36b69-istiod-ca-cert\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.091739 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.091713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-envoy\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.092075 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.092059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58136754-4a10-4398-bd0b-e869b8a36b69-istio-podinfo\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.096758 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.096734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-istio-token\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.096974 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.096956    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb6qq\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-kube-api-access-xb6qq\") pod \"e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") " pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.200332 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.200280    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:25.474278 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.474254    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"]
Sep 09 13:04:25.475763 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:04:25.475738    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58136754_4a10_4398_bd0b_e869b8a36b69.slice/crio-716ce49af4354c6443198a69de6d7bb0378e5c70ad0a6b9e24d808879c492abf WatchSource:0}: Error finding container 716ce49af4354c6443198a69de6d7bb0378e5c70ad0a6b9e24d808879c492abf: Status 404 returned error can't find the container with id 716ce49af4354c6443198a69de6d7bb0378e5c70ad0a6b9e24d808879c492abf
Sep 09 13:04:25.477923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.477886    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:04:25.478013 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.477953    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:04:25.478013 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.477980    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:04:25.571751 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.571718    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" event={"ID":"58136754-4a10-4398-bd0b-e869b8a36b69","Type":"ContainerStarted","Data":"64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d"}
Sep 09 13:04:25.571891 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.571766    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" event={"ID":"58136754-4a10-4398-bd0b-e869b8a36b69","Type":"ContainerStarted","Data":"716ce49af4354c6443198a69de6d7bb0378e5c70ad0a6b9e24d808879c492abf"}
Sep 09 13:04:25.591273 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:25.591226    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" podStartSLOduration=1.5912113190000001 podStartE2EDuration="1.591211319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:04:25.589241921 +0000 UTC m=+951.589908021" watchObservedRunningTime="2026-09-09 13:04:25.591211319 +0000 UTC m=+951.591877419"
Sep 09 13:04:26.200576 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:26.200545    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:26.205589 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:26.205567    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:26.515794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:26.515723    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ae8b19e-ceff-4137-9d06-3f4f2bdc2747" path="/var/lib/kubelet/pods/2ae8b19e-ceff-4137-9d06-3f4f2bdc2747/volumes"
Sep 09 13:04:26.516139 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:26.516125    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fb27dfe-ad0d-4e25-ac34-170599b61535" path="/var/lib/kubelet/pods/9fb27dfe-ad0d-4e25-ac34-170599b61535/volumes"
Sep 09 13:04:26.516471 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:26.516457    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a43efd7b-72c0-41e8-8048-0eb98cb4cd58" path="/var/lib/kubelet/pods/a43efd7b-72c0-41e8-8048-0eb98cb4cd58/volumes"
Sep 09 13:04:26.576481 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:26.576460    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:26.577425 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:26.577400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:28.059206 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.059176    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"]
Sep 09 13:04:28.063876 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.063855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.067175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.067155    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0175d9d3-serving-cert\""
Sep 09 13:04:28.073876 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.073849    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"]
Sep 09 13:04:28.115926 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.115899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.116034 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.115932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-metrics-certs\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.116034 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.115960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl2nr\" (UniqueName: \"kubernetes.io/projected/708fc986-104a-4517-aef2-0b09990ae76a-kube-api-access-sl2nr\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.116140 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.116078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/708fc986-104a-4517-aef2-0b09990ae76a-openshift-service-ca\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.217027 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.216991    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-metrics-certs\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.217195 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.217038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2nr\" (UniqueName: \"kubernetes.io/projected/708fc986-104a-4517-aef2-0b09990ae76a-kube-api-access-sl2nr\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.217195 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.217101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/708fc986-104a-4517-aef2-0b09990ae76a-openshift-service-ca\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.217195 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.217132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.217302 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:04:28.217227    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-serving-cert: secret "maas-api-e2e-ait-0175d9d3-serving-cert" not found
Sep 09 13:04:28.217302 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:04:28.217296    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls podName:708fc986-104a-4517-aef2-0b09990ae76a nodeName:}" failed. No retries permitted until 2026-09-09 13:04:28.717275133 +0000 UTC m=+954.717941227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls") pod "maas-api-e2e-ait-0175d9d3-857578586b-gcwqx" (UID: "708fc986-104a-4517-aef2-0b09990ae76a") : secret "maas-api-e2e-ait-0175d9d3-serving-cert" not found
Sep 09 13:04:28.217912 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.217877    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/708fc986-104a-4517-aef2-0b09990ae76a-openshift-service-ca\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.219603 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.219581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-metrics-certs\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.226024 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.226003    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl2nr\" (UniqueName: \"kubernetes.io/projected/708fc986-104a-4517-aef2-0b09990ae76a-kube-api-access-sl2nr\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.722135 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:28.722106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:28.722336 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:04:28.722252    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-serving-cert: secret "maas-api-e2e-ait-0175d9d3-serving-cert" not found
Sep 09 13:04:28.722406 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:04:28.722336    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls podName:708fc986-104a-4517-aef2-0b09990ae76a nodeName:}" failed. No retries permitted until 2026-09-09 13:04:29.72231637 +0000 UTC m=+955.722982462 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls") pod "maas-api-e2e-ait-0175d9d3-857578586b-gcwqx" (UID: "708fc986-104a-4517-aef2-0b09990ae76a") : secret "maas-api-e2e-ait-0175d9d3-serving-cert" not found
Sep 09 13:04:29.730626 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:29.730584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:29.733067 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:29.733044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls\") pod \"maas-api-e2e-ait-0175d9d3-857578586b-gcwqx\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:29.878230 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:29.878202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:30.150770 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:30.150674    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"]
Sep 09 13:04:30.156274 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:04:30.156245    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708fc986_104a_4517_aef2_0b09990ae76a.slice/crio-e578691467df36a1328effc3ce1300c8085ca67ee86d60e9962856221b26d645 WatchSource:0}: Error finding container e578691467df36a1328effc3ce1300c8085ca67ee86d60e9962856221b26d645: Status 404 returned error can't find the container with id e578691467df36a1328effc3ce1300c8085ca67ee86d60e9962856221b26d645
Sep 09 13:04:30.158155 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:30.158134    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:04:30.598686 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:30.598619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx" event={"ID":"708fc986-104a-4517-aef2-0b09990ae76a","Type":"ContainerStarted","Data":"e578691467df36a1328effc3ce1300c8085ca67ee86d60e9962856221b26d645"}
Sep 09 13:04:31.605972 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:31.605933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx" event={"ID":"708fc986-104a-4517-aef2-0b09990ae76a","Type":"ContainerStarted","Data":"842f3dc6c1939fade29f03c5968d7b65665045c170752cfbca855e6505b8ef9f"}
Sep 09 13:04:31.606354 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:31.606019    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:31.622336 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:31.622288    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx" podStartSLOduration=3.182840299 podStartE2EDuration="3.622274802s" totalImagesPullingTime="439.434503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:04:31.622048904 +0000 UTC m=+957.622715003" watchObservedRunningTime="2026-09-09 13:04:31.622274802 +0000 UTC m=+957.622940973"
Sep 09 13:04:37.618611 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:37.618575    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:45.681950 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:45.681908    2577 generic.go:356] "Generic (PLEG): container finished" podID="58136754-4a10-4398-bd0b-e869b8a36b69" containerID="64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d" exitCode=0
Sep 09 13:04:45.682405 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:45.682015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" event={"ID":"58136754-4a10-4398-bd0b-e869b8a36b69","Type":"ContainerDied","Data":"64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d"}
Sep 09 13:04:45.682405 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:45.682385    2577 scope.go:117] "RemoveContainer" containerID="64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d"
Sep 09 13:04:45.683557 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:45.683526    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:04:45.683698 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:45.683605    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:04:45.683698 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:45.683679    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:04:46.693459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:46.693414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" event={"ID":"58136754-4a10-4398-bd0b-e869b8a36b69","Type":"ContainerStarted","Data":"b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c"}
Sep 09 13:04:47.200459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:47.200421    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:47.200663 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:47.200587    2577 patch_prober.go:28] interesting pod/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 13:04:47.200718 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:47.200655    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 13:04:48.201084 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:48.201045    2577 patch_prober.go:28] interesting pod/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 13:04:48.201466 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:48.201133    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 13:04:49.201108 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:49.201081    2577 patch_prober.go:28] interesting pod/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 13:04:49.201557 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:49.201533    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 13:04:50.200836 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:50.200780    2577 patch_prober.go:28] interesting pod/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 13:04:50.201008 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:50.200854    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 13:04:51.201621 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:51.201584    2577 patch_prober.go:28] interesting pod/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 09 13:04:51.202094 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:51.201672    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 09 13:04:52.205059 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:52.205023    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:52.205527 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:52.205397    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:52.205954 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:52.205935    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:04:58.639404 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.639332    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"]
Sep 09 13:04:58.639811 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.639556    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx" podUID="708fc986-104a-4517-aef2-0b09990ae76a" containerName="maas-api" containerID="cri-o://842f3dc6c1939fade29f03c5968d7b65665045c170752cfbca855e6505b8ef9f" gracePeriod=30
Sep 09 13:04:58.755733 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.755705    2577 generic.go:356] "Generic (PLEG): container finished" podID="708fc986-104a-4517-aef2-0b09990ae76a" containerID="842f3dc6c1939fade29f03c5968d7b65665045c170752cfbca855e6505b8ef9f" exitCode=0
Sep 09 13:04:58.755840 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.755784    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx" event={"ID":"708fc986-104a-4517-aef2-0b09990ae76a","Type":"ContainerDied","Data":"842f3dc6c1939fade29f03c5968d7b65665045c170752cfbca855e6505b8ef9f"}
Sep 09 13:04:58.886188 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.886165    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:58.993298 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.993232    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls\") pod \"708fc986-104a-4517-aef2-0b09990ae76a\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") "
Sep 09 13:04:58.993298 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.993279    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/708fc986-104a-4517-aef2-0b09990ae76a-openshift-service-ca\") pod \"708fc986-104a-4517-aef2-0b09990ae76a\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") "
Sep 09 13:04:58.993499 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.993314    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-metrics-certs\") pod \"708fc986-104a-4517-aef2-0b09990ae76a\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") "
Sep 09 13:04:58.993499 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.993377    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl2nr\" (UniqueName: \"kubernetes.io/projected/708fc986-104a-4517-aef2-0b09990ae76a-kube-api-access-sl2nr\") pod \"708fc986-104a-4517-aef2-0b09990ae76a\" (UID: \"708fc986-104a-4517-aef2-0b09990ae76a\") "
Sep 09 13:04:58.993756 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.993731    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/708fc986-104a-4517-aef2-0b09990ae76a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "708fc986-104a-4517-aef2-0b09990ae76a" (UID: "708fc986-104a-4517-aef2-0b09990ae76a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:04:58.995720 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.995690    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "708fc986-104a-4517-aef2-0b09990ae76a" (UID: "708fc986-104a-4517-aef2-0b09990ae76a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:04:58.995827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.995749    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "708fc986-104a-4517-aef2-0b09990ae76a" (UID: "708fc986-104a-4517-aef2-0b09990ae76a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:04:58.995827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:58.995797    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/708fc986-104a-4517-aef2-0b09990ae76a-kube-api-access-sl2nr" (OuterVolumeSpecName: "kube-api-access-sl2nr") pod "708fc986-104a-4517-aef2-0b09990ae76a" (UID: "708fc986-104a-4517-aef2-0b09990ae76a"). InnerVolumeSpecName "kube-api-access-sl2nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:04:59.095129 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.095096    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:59.095129 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.095123    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/708fc986-104a-4517-aef2-0b09990ae76a-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:59.095129 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.095132    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/708fc986-104a-4517-aef2-0b09990ae76a-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:59.095350 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.095142    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl2nr\" (UniqueName: \"kubernetes.io/projected/708fc986-104a-4517-aef2-0b09990ae76a-kube-api-access-sl2nr\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:04:59.762566 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.762534    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"
Sep 09 13:04:59.762991 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.762573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx" event={"ID":"708fc986-104a-4517-aef2-0b09990ae76a","Type":"ContainerDied","Data":"e578691467df36a1328effc3ce1300c8085ca67ee86d60e9962856221b26d645"}
Sep 09 13:04:59.762991 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.762630    2577 scope.go:117] "RemoveContainer" containerID="842f3dc6c1939fade29f03c5968d7b65665045c170752cfbca855e6505b8ef9f"
Sep 09 13:04:59.783759 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.783728    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"]
Sep 09 13:04:59.785683 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:04:59.785657    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0175d9d3-857578586b-gcwqx"]
Sep 09 13:05:00.069895 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.069821    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"]
Sep 09 13:05:00.070103 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.070072    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy" containerID="cri-o://b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c" gracePeriod=30
Sep 09 13:05:00.514860 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.514829    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="708fc986-104a-4517-aef2-0b09990ae76a" path="/var/lib/kubelet/pods/708fc986-104a-4517-aef2-0b09990ae76a/volumes"
Sep 09 13:05:00.857075 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.857004    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"]
Sep 09 13:05:00.857570 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.857550    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="708fc986-104a-4517-aef2-0b09990ae76a" containerName="maas-api"
Sep 09 13:05:00.857615 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.857574    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="708fc986-104a-4517-aef2-0b09990ae76a" containerName="maas-api"
Sep 09 13:05:00.857708 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.857695    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="708fc986-104a-4517-aef2-0b09990ae76a" containerName="maas-api"
Sep 09 13:05:00.860954 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.860934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:00.863602 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.863581    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9b2cda79-openshift-default-dockercfg-lgdxw\""
Sep 09 13:05:00.869768 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:00.869746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"]
Sep 09 13:05:01.008835 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.008805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-data\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.009016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.008845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w2g9\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-kube-api-access-7w2g9\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.009016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.008878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istiod-ca-cert\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.009016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.008942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-envoy\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.009016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.008987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-socket\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.009016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.009009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-token\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.009255 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.009030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-podinfo\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.009255 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.009058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-certs\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.009255 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.009152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-credential-socket\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110450 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-podinfo\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110450 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-certs\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110450 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-credential-socket\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110740 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-data\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110740 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w2g9\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-kube-api-access-7w2g9\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110740 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istiod-ca-cert\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110740 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-envoy\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110740 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-socket\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.110740 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-token\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.111033 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110945    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-credential-socket\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.111033 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.110975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-certs\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.111145 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.111085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-socket\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.111200 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.111169    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-data\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.111374 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.111352    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istiod-ca-cert\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.113053 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.113034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-envoy\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.113191 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.113172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-podinfo\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.118039 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.118018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-token\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.118252 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.118234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w2g9\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-kube-api-access-7w2g9\") pod \"e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") " pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.174396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.174376    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:01.449323 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.449300    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"]
Sep 09 13:05:01.450459 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:05:01.450433    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30295d3_1df3_4935_9a2f_c158d3d7f16e.slice/crio-acdd1cfcbf22fcdbaab27a9f4888c214e87497d19f31212a4c0454648d8f8ec7 WatchSource:0}: Error finding container acdd1cfcbf22fcdbaab27a9f4888c214e87497d19f31212a4c0454648d8f8ec7: Status 404 returned error can't find the container with id acdd1cfcbf22fcdbaab27a9f4888c214e87497d19f31212a4c0454648d8f8ec7
Sep 09 13:05:01.452815 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.452784    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:05:01.452902 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.452846    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:05:01.452902 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.452893    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:05:01.778265 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.778162    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" event={"ID":"e30295d3-1df3-4935-9a2f-c158d3d7f16e","Type":"ContainerStarted","Data":"a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c"}
Sep 09 13:05:01.778265 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.778230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" event={"ID":"e30295d3-1df3-4935-9a2f-c158d3d7f16e","Type":"ContainerStarted","Data":"acdd1cfcbf22fcdbaab27a9f4888c214e87497d19f31212a4c0454648d8f8ec7"}
Sep 09 13:05:01.796763 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:01.796707    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" podStartSLOduration=1.7966876790000001 podStartE2EDuration="1.796687679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:05:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:05:01.796299071 +0000 UTC m=+987.796965173" watchObservedRunningTime="2026-09-09 13:05:01.796687679 +0000 UTC m=+987.797353780"
Sep 09 13:05:02.174858 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:02.174829    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:02.179891 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:02.179870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:02.783251 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:02.783218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:02.784531 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:02.784508    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:04.920307 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:04.920268    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"]
Sep 09 13:05:04.923755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:04.923738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:04.926659 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:04.926605    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-9b2cda79\""
Sep 09 13:05:04.926866 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:04.926847    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-9b2cda79-dockercfg-ff85x\""
Sep 09 13:05:04.931754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:04.931397    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"]
Sep 09 13:05:05.046528 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.046490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gprgt\" (UniqueName: \"kubernetes.io/projected/aee7a130-ba02-40d6-b4ae-a74d3059656b-kube-api-access-gprgt\") pod \"payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n\" (UID: \"aee7a130-ba02-40d6-b4ae-a74d3059656b\") " pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:05.046528 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.046530    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aee7a130-ba02-40d6-b4ae-a74d3059656b-plugins-config-volume\") pod \"payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n\" (UID: \"aee7a130-ba02-40d6-b4ae-a74d3059656b\") " pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:05.147941 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.147911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gprgt\" (UniqueName: \"kubernetes.io/projected/aee7a130-ba02-40d6-b4ae-a74d3059656b-kube-api-access-gprgt\") pod \"payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n\" (UID: \"aee7a130-ba02-40d6-b4ae-a74d3059656b\") " pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:05.148053 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.147946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aee7a130-ba02-40d6-b4ae-a74d3059656b-plugins-config-volume\") pod \"payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n\" (UID: \"aee7a130-ba02-40d6-b4ae-a74d3059656b\") " pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:05.148690 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.148672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aee7a130-ba02-40d6-b4ae-a74d3059656b-plugins-config-volume\") pod \"payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n\" (UID: \"aee7a130-ba02-40d6-b4ae-a74d3059656b\") " pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:05.155833 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.155812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gprgt\" (UniqueName: \"kubernetes.io/projected/aee7a130-ba02-40d6-b4ae-a74d3059656b-kube-api-access-gprgt\") pod \"payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n\" (UID: \"aee7a130-ba02-40d6-b4ae-a74d3059656b\") " pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:05.235800 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.235772    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:05.332921 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.332899    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:05:05.451281 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451217    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-certs\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451281 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451249    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-data\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451281 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451267    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-credential-socket\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-socket\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451307    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-istio-token\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451323    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58136754-4a10-4398-bd0b-e869b8a36b69-istio-podinfo\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-envoy\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451488    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb6qq\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-kube-api-access-xb6qq\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451500    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:05.451539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451513    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-data" (OuterVolumeSpecName: "istio-data") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:05.451539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451533    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58136754-4a10-4398-bd0b-e869b8a36b69-istiod-ca-cert\") pod \"58136754-4a10-4398-bd0b-e869b8a36b69\" (UID: \"58136754-4a10-4398-bd0b-e869b8a36b69\") "
Sep 09 13:05:05.451938 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451867    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.451938 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.451887    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.452080 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.452062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:05.452135 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.452077    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:05.452382 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.452359    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58136754-4a10-4398-bd0b-e869b8a36b69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:05:05.454172 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.454124    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58136754-4a10-4398-bd0b-e869b8a36b69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:05:05.454270 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.454191    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-kube-api-access-xb6qq" (OuterVolumeSpecName: "kube-api-access-xb6qq") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "kube-api-access-xb6qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:05:05.454270 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.454202    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:05.454270 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.454209    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-istio-token" (OuterVolumeSpecName: "istio-token") pod "58136754-4a10-4398-bd0b-e869b8a36b69" (UID: "58136754-4a10-4398-bd0b-e869b8a36b69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:05:05.513417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.513399    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"]
Sep 09 13:05:05.515454 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:05:05.515435    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee7a130_ba02_40d6_b4ae_a74d3059656b.slice/crio-a02902279a6df9c48a30585593b1b9fd465caeca77b7a408ae48a1c14d956366 WatchSource:0}: Error finding container a02902279a6df9c48a30585593b1b9fd465caeca77b7a408ae48a1c14d956366: Status 404 returned error can't find the container with id a02902279a6df9c48a30585593b1b9fd465caeca77b7a408ae48a1c14d956366
Sep 09 13:05:05.557623 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.557597    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.557758 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.557628    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.557758 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.557670    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.557758 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.557683    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58136754-4a10-4398-bd0b-e869b8a36b69-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.557758 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.557694    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58136754-4a10-4398-bd0b-e869b8a36b69-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.557758 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.557707    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb6qq\" (UniqueName: \"kubernetes.io/projected/58136754-4a10-4398-bd0b-e869b8a36b69-kube-api-access-xb6qq\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.557758 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.557729    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58136754-4a10-4398-bd0b-e869b8a36b69-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:05.802660 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.802545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n" event={"ID":"aee7a130-ba02-40d6-b4ae-a74d3059656b","Type":"ContainerStarted","Data":"30ae4f9abcbad058d54fcf68bc25412ed47e1c5306ccb472259c95a56e1c7f7c"}
Sep 09 13:05:05.802660 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.802612    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:05.802660 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.802654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n" event={"ID":"aee7a130-ba02-40d6-b4ae-a74d3059656b","Type":"ContainerStarted","Data":"a02902279a6df9c48a30585593b1b9fd465caeca77b7a408ae48a1c14d956366"}
Sep 09 13:05:05.804298 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.804273    2577 generic.go:356] "Generic (PLEG): container finished" podID="58136754-4a10-4398-bd0b-e869b8a36b69" containerID="b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c" exitCode=0
Sep 09 13:05:05.804421 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.804321    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"
Sep 09 13:05:05.804421 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.804316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" event={"ID":"58136754-4a10-4398-bd0b-e869b8a36b69","Type":"ContainerDied","Data":"b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c"}
Sep 09 13:05:05.804501 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.804437    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm" event={"ID":"58136754-4a10-4398-bd0b-e869b8a36b69","Type":"ContainerDied","Data":"716ce49af4354c6443198a69de6d7bb0378e5c70ad0a6b9e24d808879c492abf"}
Sep 09 13:05:05.804501 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.804460    2577 scope.go:117] "RemoveContainer" containerID="b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c"
Sep 09 13:05:05.813149 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.813130    2577 scope.go:117] "RemoveContainer" containerID="64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d"
Sep 09 13:05:05.819253 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.819215    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n" podStartSLOduration=1.819203387 podStartE2EDuration="1.819203387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:05:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:05:05.817450613 +0000 UTC m=+991.818116712" watchObservedRunningTime="2026-09-09 13:05:05.819203387 +0000 UTC m=+991.819869487"
Sep 09 13:05:05.822999 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.822985    2577 scope.go:117] "RemoveContainer" containerID="b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c"
Sep 09 13:05:05.823276 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:05:05.823258    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c\": container with ID starting with b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c not found: ID does not exist" containerID="b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c"
Sep 09 13:05:05.823333 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.823283    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c"} err="failed to get container status \"b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c\": rpc error: code = NotFound desc = could not find container \"b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c\": container with ID starting with b9a3ac9a33ab16fae2549e04b616e433c976bbca1186bc805800eb38745dac5c not found: ID does not exist"
Sep 09 13:05:05.823333 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.823297    2577 scope.go:117] "RemoveContainer" containerID="64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d"
Sep 09 13:05:05.823560 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:05:05.823538    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d\": container with ID starting with 64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d not found: ID does not exist" containerID="64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d"
Sep 09 13:05:05.823601 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.823572    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d"} err="failed to get container status \"64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d\": rpc error: code = NotFound desc = could not find container \"64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d\": container with ID starting with 64d8e0c09e2937be6158b0f751c31ccc7d3f1a7d25c19da4f3855afd59be8f4d not found: ID does not exist"
Sep 09 13:05:05.831048 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.831027    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"]
Sep 09 13:05:05.834727 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:05.834709    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0175d9d3-openshift-default-754b4f5bf8-r2hqm"]
Sep 09 13:05:06.516721 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:06.516681    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" path="/var/lib/kubelet/pods/58136754-4a10-4398-bd0b-e869b8a36b69/volumes"
Sep 09 13:05:16.812496 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:16.812462    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:20.849798 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:20.849764    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"]
Sep 09 13:05:20.850253 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:20.850014    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx" podUID="cdb3b7ef-972f-4ade-9987-c9849c787ed0" containerName="payload-pre-processing" containerID="cri-o://9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd" gracePeriod=30
Sep 09 13:05:21.110424 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.110291    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:05:21.186074 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.186011    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-952dw\" (UniqueName: \"kubernetes.io/projected/cdb3b7ef-972f-4ade-9987-c9849c787ed0-kube-api-access-952dw\") pod \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\" (UID: \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\") "
Sep 09 13:05:21.186205 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.186075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdb3b7ef-972f-4ade-9987-c9849c787ed0-plugins-config-volume\") pod \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\" (UID: \"cdb3b7ef-972f-4ade-9987-c9849c787ed0\") "
Sep 09 13:05:21.186430 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.186404    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdb3b7ef-972f-4ade-9987-c9849c787ed0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdb3b7ef-972f-4ade-9987-c9849c787ed0" (UID: "cdb3b7ef-972f-4ade-9987-c9849c787ed0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:05:21.188242 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.188208    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdb3b7ef-972f-4ade-9987-c9849c787ed0-kube-api-access-952dw" (OuterVolumeSpecName: "kube-api-access-952dw") pod "cdb3b7ef-972f-4ade-9987-c9849c787ed0" (UID: "cdb3b7ef-972f-4ade-9987-c9849c787ed0"). InnerVolumeSpecName "kube-api-access-952dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:05:21.287525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.287491    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-952dw\" (UniqueName: \"kubernetes.io/projected/cdb3b7ef-972f-4ade-9987-c9849c787ed0-kube-api-access-952dw\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:21.287525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.287520    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdb3b7ef-972f-4ade-9987-c9849c787ed0-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:21.884226 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.884187    2577 generic.go:356] "Generic (PLEG): container finished" podID="cdb3b7ef-972f-4ade-9987-c9849c787ed0" containerID="9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd" exitCode=0
Sep 09 13:05:21.884777 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.884234    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"
Sep 09 13:05:21.884777 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.884281    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx" event={"ID":"cdb3b7ef-972f-4ade-9987-c9849c787ed0","Type":"ContainerDied","Data":"9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd"}
Sep 09 13:05:21.884777 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.884342    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx" event={"ID":"cdb3b7ef-972f-4ade-9987-c9849c787ed0","Type":"ContainerDied","Data":"c29be6e0e4adb866c6212f43ce0702f87b02c496af9aa25863f3a632b31b96ae"}
Sep 09 13:05:21.884777 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.884375    2577 scope.go:117] "RemoveContainer" containerID="9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd"
Sep 09 13:05:21.886227 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.886199    2577 generic.go:356] "Generic (PLEG): container finished" podID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerID="a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c" exitCode=0
Sep 09 13:05:21.886327 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.886248    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" event={"ID":"e30295d3-1df3-4935-9a2f-c158d3d7f16e","Type":"ContainerDied","Data":"a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c"}
Sep 09 13:05:21.886612 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.886599    2577 scope.go:117] "RemoveContainer" containerID="a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c"
Sep 09 13:05:21.887856 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.887825    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:05:21.887965 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.887909    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:05:21.887965 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.887950    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:05:21.893848 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.893832    2577 scope.go:117] "RemoveContainer" containerID="9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd"
Sep 09 13:05:21.894126 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:05:21.894109    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd\": container with ID starting with 9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd not found: ID does not exist" containerID="9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd"
Sep 09 13:05:21.894173 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.894135    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd"} err="failed to get container status \"9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd\": rpc error: code = NotFound desc = could not find container \"9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd\": container with ID starting with 9f0fb1e677eb46275cedddaaf5003aad6a11110c9e58369d44be7d5593941abd not found: ID does not exist"
Sep 09 13:05:21.921778 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.921754    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"]
Sep 09 13:05:21.929082 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:21.929056    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b4ea80-5799b546f4-gskpx"]
Sep 09 13:05:22.515329 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:22.515285    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdb3b7ef-972f-4ade-9987-c9849c787ed0" path="/var/lib/kubelet/pods/cdb3b7ef-972f-4ade-9987-c9849c787ed0/volumes"
Sep 09 13:05:22.895873 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:22.895840    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" event={"ID":"e30295d3-1df3-4935-9a2f-c158d3d7f16e","Type":"ContainerStarted","Data":"d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec"}
Sep 09 13:05:23.175423 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:23.175323    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:23.175963 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:23.175938    2577 patch_prober.go:28] interesting pod/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 09 13:05:23.176150 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:23.176125    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 09 13:05:24.176217 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.175778    2577 patch_prober.go:28] interesting pod/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 09 13:05:24.176217 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.175839    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 09 13:05:24.377065 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.376990    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"]
Sep 09 13:05:24.377993 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.377973    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy"
Sep 09 13:05:24.378127 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.378116    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy"
Sep 09 13:05:24.378241 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.378229    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy"
Sep 09 13:05:24.378313 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.378304    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy"
Sep 09 13:05:24.378404 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.378395    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdb3b7ef-972f-4ade-9987-c9849c787ed0" containerName="payload-pre-processing"
Sep 09 13:05:24.378467 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.378459    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdb3b7ef-972f-4ade-9987-c9849c787ed0" containerName="payload-pre-processing"
Sep 09 13:05:24.378608 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.378598    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy"
Sep 09 13:05:24.378715 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.378705    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdb3b7ef-972f-4ade-9987-c9849c787ed0" containerName="payload-pre-processing"
Sep 09 13:05:24.378789 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.378781    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="58136754-4a10-4398-bd0b-e869b8a36b69" containerName="istio-proxy"
Sep 09 13:05:24.392059 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.392039    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"]
Sep 09 13:05:24.392738 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.392719    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.402016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.401989    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3b4b96\"/\"default-dockercfg-gx2nk\""
Sep 09 13:05:24.402201 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.402172    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3b4b96\"/\"test-model-3b4b96-kserve-self-signed-certs\""
Sep 09 13:05:24.402404 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.402385    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3b4b96\"/\"openshift-service-ca.crt\""
Sep 09 13:05:24.402581 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.402548    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3b4b96\"/\"kube-root-ca.crt\""
Sep 09 13:05:24.416072 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.416040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tmp-dir\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.416175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.416095    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz62v\" (UniqueName: \"kubernetes.io/projected/04b1e7be-4656-42aa-a6a3-cfd591a32f12-kube-api-access-nz62v\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.416175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.416148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-home\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.416283 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.416262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-model-cache\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.416342 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.416291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tls-certs\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.416393 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.416339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-dshm\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.517075 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.516976    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-model-cache\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.517075 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.517023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tls-certs\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.517075 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.517065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-dshm\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.517614 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.517091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tmp-dir\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.517614 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.517125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz62v\" (UniqueName: \"kubernetes.io/projected/04b1e7be-4656-42aa-a6a3-cfd591a32f12-kube-api-access-nz62v\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.517614 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.517160    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-home\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.517614 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.517442    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-model-cache\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.517614 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.517489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tmp-dir\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.518090 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.517782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-home\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.520382 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.520353    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tls-certs\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.520490 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.520418    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-dshm\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.524547 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.524521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz62v\" (UniqueName: \"kubernetes.io/projected/04b1e7be-4656-42aa-a6a3-cfd591a32f12-kube-api-access-nz62v\") pod \"test-model-3b4b96-kserve-6b5c4cb9df-jzp8f\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") " pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.710459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.710419    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:24.985588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:24.985543    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"]
Sep 09 13:05:24.997400 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:05:24.997373    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04b1e7be_4656_42aa_a6a3_cfd591a32f12.slice/crio-e8c71b221553d72c5b642630479b736e926df0d8e80bafe6b56d0af196fec797 WatchSource:0}: Error finding container e8c71b221553d72c5b642630479b736e926df0d8e80bafe6b56d0af196fec797: Status 404 returned error can't find the container with id e8c71b221553d72c5b642630479b736e926df0d8e80bafe6b56d0af196fec797
Sep 09 13:05:25.175397 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:25.175357    2577 patch_prober.go:28] interesting pod/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 09 13:05:25.175661 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:25.175429    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 09 13:05:25.912690 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:25.912652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f" event={"ID":"04b1e7be-4656-42aa-a6a3-cfd591a32f12","Type":"ContainerStarted","Data":"036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424"}
Sep 09 13:05:25.912690 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:25.912695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f" event={"ID":"04b1e7be-4656-42aa-a6a3-cfd591a32f12","Type":"ContainerStarted","Data":"e8c71b221553d72c5b642630479b736e926df0d8e80bafe6b56d0af196fec797"}
Sep 09 13:05:25.930281 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:25.930235    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f" podStartSLOduration=1.9302208379999999 podStartE2EDuration="1.930220838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:05:25.927677609 +0000 UTC m=+1011.928343711" watchObservedRunningTime="2026-09-09 13:05:25.930220838 +0000 UTC m=+1011.930886939"
Sep 09 13:05:26.175006 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:26.174917    2577 patch_prober.go:28] interesting pod/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 09 13:05:26.175261 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:26.175231    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 09 13:05:27.175867 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:27.175827    2577 patch_prober.go:28] interesting pod/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Sep 09 13:05:27.176349 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:27.175888    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Sep 09 13:05:28.208410 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:28.208380    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:28.208849 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:28.208813    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:28.210005 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:28.209983    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:34.711005 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:34.710973    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:34.711395 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:34.711039    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:34.723900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:34.723832    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:34.810749 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:34.810504    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"]
Sep 09 13:05:34.811116 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:34.811086    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n" podUID="aee7a130-ba02-40d6-b4ae-a74d3059656b" containerName="payload-processing" containerID="cri-o://30ae4f9abcbad058d54fcf68bc25412ed47e1c5306ccb472259c95a56e1c7f7c" gracePeriod=30
Sep 09 13:05:34.959248 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:34.959211    2577 generic.go:356] "Generic (PLEG): container finished" podID="aee7a130-ba02-40d6-b4ae-a74d3059656b" containerID="30ae4f9abcbad058d54fcf68bc25412ed47e1c5306ccb472259c95a56e1c7f7c" exitCode=0
Sep 09 13:05:34.959398 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:34.959282    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n" event={"ID":"aee7a130-ba02-40d6-b4ae-a74d3059656b","Type":"ContainerDied","Data":"30ae4f9abcbad058d54fcf68bc25412ed47e1c5306ccb472259c95a56e1c7f7c"}
Sep 09 13:05:34.975761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:34.975686    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:05:35.060426 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.060404    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:35.113016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.112984    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aee7a130-ba02-40d6-b4ae-a74d3059656b-plugins-config-volume\") pod \"aee7a130-ba02-40d6-b4ae-a74d3059656b\" (UID: \"aee7a130-ba02-40d6-b4ae-a74d3059656b\") "
Sep 09 13:05:35.113016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.113023    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gprgt\" (UniqueName: \"kubernetes.io/projected/aee7a130-ba02-40d6-b4ae-a74d3059656b-kube-api-access-gprgt\") pod \"aee7a130-ba02-40d6-b4ae-a74d3059656b\" (UID: \"aee7a130-ba02-40d6-b4ae-a74d3059656b\") "
Sep 09 13:05:35.113396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.113370    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee7a130-ba02-40d6-b4ae-a74d3059656b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aee7a130-ba02-40d6-b4ae-a74d3059656b" (UID: "aee7a130-ba02-40d6-b4ae-a74d3059656b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:05:35.115853 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.115824    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee7a130-ba02-40d6-b4ae-a74d3059656b-kube-api-access-gprgt" (OuterVolumeSpecName: "kube-api-access-gprgt") pod "aee7a130-ba02-40d6-b4ae-a74d3059656b" (UID: "aee7a130-ba02-40d6-b4ae-a74d3059656b"). InnerVolumeSpecName "kube-api-access-gprgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:05:35.122249 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.122220    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"]
Sep 09 13:05:35.122486 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.122461    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy" containerID="cri-o://d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec" gracePeriod=30
Sep 09 13:05:35.215054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.215015    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aee7a130-ba02-40d6-b4ae-a74d3059656b-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:35.215312 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.215294    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gprgt\" (UniqueName: \"kubernetes.io/projected/aee7a130-ba02-40d6-b4ae-a74d3059656b-kube-api-access-gprgt\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:35.965017 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.964988    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"
Sep 09 13:05:35.965465 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.965021    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n" event={"ID":"aee7a130-ba02-40d6-b4ae-a74d3059656b","Type":"ContainerDied","Data":"a02902279a6df9c48a30585593b1b9fd465caeca77b7a408ae48a1c14d956366"}
Sep 09 13:05:35.965465 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.965087    2577 scope.go:117] "RemoveContainer" containerID="30ae4f9abcbad058d54fcf68bc25412ed47e1c5306ccb472259c95a56e1c7f7c"
Sep 09 13:05:35.985272 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.985249    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"]
Sep 09 13:05:35.989175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:35.989150    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9b2cda79-7599bd9677-ng82n"]
Sep 09 13:05:36.515556 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:36.515524    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aee7a130-ba02-40d6-b4ae-a74d3059656b" path="/var/lib/kubelet/pods/aee7a130-ba02-40d6-b4ae-a74d3059656b/volumes"
Sep 09 13:05:40.372270 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.372248    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:40.453940 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.453878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-token\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.453940 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.453912    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-certs\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.453940 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.453931    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istiod-ca-cert\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.454205 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.453947    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-credential-socket\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.454205 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.453968    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-podinfo\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.454205 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454012    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w2g9\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-kube-api-access-7w2g9\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.454205 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454026    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-envoy\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.454205 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454061    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-data\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.454205 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-socket\") pod \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\" (UID: \"e30295d3-1df3-4935-9a2f-c158d3d7f16e\") "
Sep 09 13:05:40.454503 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454256    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:40.454503 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:40.454503 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454323    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:05:40.454503 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454496    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:40.454745 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454505    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-data" (OuterVolumeSpecName: "istio-data") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:40.454745 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454547    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.454745 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454567    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.454745 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.454580    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.456375 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.456351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:05:40.456485 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.456459    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-kube-api-access-7w2g9" (OuterVolumeSpecName: "kube-api-access-7w2g9") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "kube-api-access-7w2g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:05:40.456747 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.456730    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-token" (OuterVolumeSpecName: "istio-token") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:05:40.456808 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.456789    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e30295d3-1df3-4935-9a2f-c158d3d7f16e" (UID: "e30295d3-1df3-4935-9a2f-c158d3d7f16e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:05:40.555227 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.555206    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.555335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.555231    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.555335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.555242    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w2g9\" (UniqueName: \"kubernetes.io/projected/e30295d3-1df3-4935-9a2f-c158d3d7f16e-kube-api-access-7w2g9\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.555335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.555251    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.555335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.555260    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.555335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.555267    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e30295d3-1df3-4935-9a2f-c158d3d7f16e-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:05:40.992761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.992732    2577 generic.go:356] "Generic (PLEG): container finished" podID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerID="d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec" exitCode=0
Sep 09 13:05:40.992900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.992782    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"
Sep 09 13:05:40.992900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.992820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" event={"ID":"e30295d3-1df3-4935-9a2f-c158d3d7f16e","Type":"ContainerDied","Data":"d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec"}
Sep 09 13:05:40.992900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.992876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj" event={"ID":"e30295d3-1df3-4935-9a2f-c158d3d7f16e","Type":"ContainerDied","Data":"acdd1cfcbf22fcdbaab27a9f4888c214e87497d19f31212a4c0454648d8f8ec7"}
Sep 09 13:05:40.992900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:40.992899    2577 scope.go:117] "RemoveContainer" containerID="d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec"
Sep 09 13:05:41.002425 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:41.002401    2577 scope.go:117] "RemoveContainer" containerID="a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c"
Sep 09 13:05:41.010402 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:41.010372    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"]
Sep 09 13:05:41.011047 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:41.011029    2577 scope.go:117] "RemoveContainer" containerID="d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec"
Sep 09 13:05:41.011317 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:05:41.011298    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec\": container with ID starting with d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec not found: ID does not exist" containerID="d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec"
Sep 09 13:05:41.011409 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:41.011336    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec"} err="failed to get container status \"d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec\": rpc error: code = NotFound desc = could not find container \"d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec\": container with ID starting with d23bc9619c8579e8dd57eab572d3851aa06d84b1df9f2a802eaa1b8447668dec not found: ID does not exist"
Sep 09 13:05:41.011409 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:41.011353    2577 scope.go:117] "RemoveContainer" containerID="a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c"
Sep 09 13:05:41.011863 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:05:41.011834    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c\": container with ID starting with a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c not found: ID does not exist" containerID="a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c"
Sep 09 13:05:41.011971 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:41.011870    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c"} err="failed to get container status \"a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c\": rpc error: code = NotFound desc = could not find container \"a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c\": container with ID starting with a1b638a3fafcf3cc9e93b74203a56f55e0be3beb2dad6fb6fe6d02ae5356d35c not found: ID does not exist"
Sep 09 13:05:41.014244 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:41.014222    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9b2cda79-openshift-default-6cddfbc996-7bsjj"]
Sep 09 13:05:42.515348 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:05:42.515313    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" path="/var/lib/kubelet/pods/e30295d3-1df3-4935-9a2f-c158d3d7f16e/volumes"
Sep 09 13:06:04.385647 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.385601    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"]
Sep 09 13:06:04.386054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.385950    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy"
Sep 09 13:06:04.386054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.385962    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy"
Sep 09 13:06:04.386054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.385979    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee7a130-ba02-40d6-b4ae-a74d3059656b" containerName="payload-processing"
Sep 09 13:06:04.386054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.385985    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee7a130-ba02-40d6-b4ae-a74d3059656b" containerName="payload-processing"
Sep 09 13:06:04.386054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.386041    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy"
Sep 09 13:06:04.386054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.386049    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee7a130-ba02-40d6-b4ae-a74d3059656b" containerName="payload-processing"
Sep 09 13:06:04.388821 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.388801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.391162 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.391144    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-c2e681-openshift-default-dockercfg-n88gh\""
Sep 09 13:06:04.398561 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.398540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"]
Sep 09 13:06:04.544711 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544678    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-socket\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.544711 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-data\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.544917 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-certs\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.544917 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-credential-socket\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.544917 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-istio-token\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.544917 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5132e023-3cd9-4846-b206-260284ffc53c-istio-podinfo\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.544917 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5132e023-3cd9-4846-b206-260284ffc53c-istiod-ca-cert\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.545072 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zxpm\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-kube-api-access-8zxpm\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.545072 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.544976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-envoy\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646145 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zxpm\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-kube-api-access-8zxpm\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646145 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-envoy\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646307 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-socket\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646346 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-data\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646394 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-certs\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646445 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-credential-socket\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646501 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-istio-token\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646501 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5132e023-3cd9-4846-b206-260284ffc53c-istio-podinfo\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646599 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646522    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5132e023-3cd9-4846-b206-260284ffc53c-istiod-ca-cert\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646599 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-socket\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-data\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-certs\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.646893 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.646869    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-credential-socket\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.647294 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.647276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5132e023-3cd9-4846-b206-260284ffc53c-istiod-ca-cert\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.648743 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.648721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-envoy\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.649219 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.649203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5132e023-3cd9-4846-b206-260284ffc53c-istio-podinfo\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.656243 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.656222    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-istio-token\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.658057 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.658027    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zxpm\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-kube-api-access-8zxpm\") pod \"e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") " pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.699366 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.699338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:04.975165 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.975140    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"]
Sep 09 13:06:04.976008 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:06:04.975983    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5132e023_3cd9_4846_b206_260284ffc53c.slice/crio-d5acae3a553dfdfde3db03942e9c663f08fedd49bf27e19496fc909fbcd5d028 WatchSource:0}: Error finding container d5acae3a553dfdfde3db03942e9c663f08fedd49bf27e19496fc909fbcd5d028: Status 404 returned error can't find the container with id d5acae3a553dfdfde3db03942e9c663f08fedd49bf27e19496fc909fbcd5d028
Sep 09 13:06:04.978320 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.978284    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:06:04.978382 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.978366    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:06:04.978431 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:04.978417    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:06:05.105485 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:05.105441    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" event={"ID":"5132e023-3cd9-4846-b206-260284ffc53c","Type":"ContainerStarted","Data":"404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33"}
Sep 09 13:06:05.105657 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:05.105548    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" event={"ID":"5132e023-3cd9-4846-b206-260284ffc53c","Type":"ContainerStarted","Data":"d5acae3a553dfdfde3db03942e9c663f08fedd49bf27e19496fc909fbcd5d028"}
Sep 09 13:06:05.130332 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:05.130287    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" podStartSLOduration=1.130271558 podStartE2EDuration="1.130271558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:06:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:06:05.128304148 +0000 UTC m=+1051.128970263" watchObservedRunningTime="2026-09-09 13:06:05.130271558 +0000 UTC m=+1051.130937658"
Sep 09 13:06:05.699446 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:05.699420    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:06.705187 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:06.705150    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:07.114385 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:07.114302    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:07.115180 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:07.115163    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:08.201448 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.201404    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"]
Sep 09 13:06:08.201967 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.201776    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy"
Sep 09 13:06:08.201967 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.201788    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy"
Sep 09 13:06:08.201967 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.201853    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e30295d3-1df3-4935-9a2f-c158d3d7f16e" containerName="istio-proxy"
Sep 09 13:06:08.204452 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.204434    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.206858 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.206839    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-c2e681-serving-cert\""
Sep 09 13:06:08.214875 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.214852    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"]
Sep 09 13:06:08.380471 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.380437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-maas-api-tls\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.380689 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.380487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ef742-dba5-40cb-9445-4d872f4766e3-openshift-service-ca\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.380689 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.380566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvsmk\" (UniqueName: \"kubernetes.io/projected/ff1ef742-dba5-40cb-9445-4d872f4766e3-kube-api-access-hvsmk\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.380689 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.380612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-metrics-certs\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.481552 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.481456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-maas-api-tls\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.481552 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.481519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ef742-dba5-40cb-9445-4d872f4766e3-openshift-service-ca\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.481810 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.481581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvsmk\" (UniqueName: \"kubernetes.io/projected/ff1ef742-dba5-40cb-9445-4d872f4766e3-kube-api-access-hvsmk\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.481810 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.481609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-metrics-certs\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.482334 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.482238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ef742-dba5-40cb-9445-4d872f4766e3-openshift-service-ca\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.484152 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.484125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-metrics-certs\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.484244 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.484172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-maas-api-tls\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.489562 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.489538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvsmk\" (UniqueName: \"kubernetes.io/projected/ff1ef742-dba5-40cb-9445-4d872f4766e3-kube-api-access-hvsmk\") pod \"maas-api-e2e-explicit-c2e681-7c86668d9b-7src7\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.517297 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.517275    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:08.787906 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.787836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"]
Sep 09 13:06:08.790862 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:06:08.790820    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1ef742_dba5_40cb_9445_4d872f4766e3.slice/crio-1afd25072891a4e3fc54e8036d85958e81093280ee8524207162f117b30b255c WatchSource:0}: Error finding container 1afd25072891a4e3fc54e8036d85958e81093280ee8524207162f117b30b255c: Status 404 returned error can't find the container with id 1afd25072891a4e3fc54e8036d85958e81093280ee8524207162f117b30b255c
Sep 09 13:06:08.816990 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.816962    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"]
Sep 09 13:06:08.823061 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.823030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:08.826259 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.826237    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-c2e681\""
Sep 09 13:06:08.826759 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.826739    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-c2e681-dockercfg-dwck7\""
Sep 09 13:06:08.832369 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.832344    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"]
Sep 09 13:06:08.986794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.986761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt\" (UID: \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:08.986794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:08.986797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzwt8\" (UniqueName: \"kubernetes.io/projected/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-kube-api-access-rzwt8\") pod \"payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt\" (UID: \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:09.088133 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:09.088035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt\" (UID: \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:09.088133 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:09.088077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzwt8\" (UniqueName: \"kubernetes.io/projected/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-kube-api-access-rzwt8\") pod \"payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt\" (UID: \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:09.088848 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:09.088827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt\" (UID: \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:09.096192 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:09.096171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzwt8\" (UniqueName: \"kubernetes.io/projected/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-kube-api-access-rzwt8\") pod \"payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt\" (UID: \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:09.127694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:09.127656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7" event={"ID":"ff1ef742-dba5-40cb-9445-4d872f4766e3","Type":"ContainerStarted","Data":"1afd25072891a4e3fc54e8036d85958e81093280ee8524207162f117b30b255c"}
Sep 09 13:06:09.136082 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:09.136054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:09.408146 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:09.408120    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"]
Sep 09 13:06:09.410308 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:06:09.410273    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cbd4ba1_22e8_4af5_8425_845e6eb2e980.slice/crio-862fd33003f4a950cd549260a5704476cf8c0a9f51295a645dca64249f595c6e WatchSource:0}: Error finding container 862fd33003f4a950cd549260a5704476cf8c0a9f51295a645dca64249f595c6e: Status 404 returned error can't find the container with id 862fd33003f4a950cd549260a5704476cf8c0a9f51295a645dca64249f595c6e
Sep 09 13:06:10.134293 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:10.134255    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7" event={"ID":"ff1ef742-dba5-40cb-9445-4d872f4766e3","Type":"ContainerStarted","Data":"a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d"}
Sep 09 13:06:10.134478 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:10.134382    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:10.135970 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:10.135946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt" event={"ID":"7cbd4ba1-22e8-4af5-8425-845e6eb2e980","Type":"ContainerStarted","Data":"9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31"}
Sep 09 13:06:10.136078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:10.135984    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt" event={"ID":"7cbd4ba1-22e8-4af5-8425-845e6eb2e980","Type":"ContainerStarted","Data":"862fd33003f4a950cd549260a5704476cf8c0a9f51295a645dca64249f595c6e"}
Sep 09 13:06:10.136078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:10.136017    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:10.151196 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:10.151150    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7" podStartSLOduration=1.717566239 podStartE2EDuration="2.151101621s" totalImagesPullingTime="433.535382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:06:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:06:10.150213316 +0000 UTC m=+1056.150879416" watchObservedRunningTime="2026-09-09 13:06:10.151101621 +0000 UTC m=+1056.151767722"
Sep 09 13:06:10.164968 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:10.164915    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt" podStartSLOduration=2.164898796 podStartE2EDuration="2.164898796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:06:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:06:10.163178919 +0000 UTC m=+1056.163845019" watchObservedRunningTime="2026-09-09 13:06:10.164898796 +0000 UTC m=+1056.165564960"
Sep 09 13:06:16.146537 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:16.146504    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:21.143327 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:21.143296    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:30.249563 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:30.249528    2577 generic.go:356] "Generic (PLEG): container finished" podID="5132e023-3cd9-4846-b206-260284ffc53c" containerID="404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33" exitCode=0
Sep 09 13:06:30.250030 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:30.249609    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" event={"ID":"5132e023-3cd9-4846-b206-260284ffc53c","Type":"ContainerDied","Data":"404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33"}
Sep 09 13:06:30.250078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:30.250048    2577 scope.go:117] "RemoveContainer" containerID="404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33"
Sep 09 13:06:30.251197 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:30.251157    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:06:30.251309 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:30.251242    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:06:30.251309 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:30.251290    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:06:31.256744 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:31.256711    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" event={"ID":"5132e023-3cd9-4846-b206-260284ffc53c","Type":"ContainerStarted","Data":"93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48"}
Sep 09 13:06:31.699845 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:31.699810    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:31.700130 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:31.700107    2577 patch_prober.go:28] interesting pod/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 09 13:06:31.700192 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:31.700172    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 09 13:06:32.700560 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:32.700517    2577 patch_prober.go:28] interesting pod/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 09 13:06:32.701017 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:32.700584    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 09 13:06:33.700766 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:33.700727    2577 patch_prober.go:28] interesting pod/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 09 13:06:33.701150 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:33.700799    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 09 13:06:34.700286 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:34.700244    2577 patch_prober.go:28] interesting pod/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 09 13:06:34.700553 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:34.700323    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 09 13:06:35.700276 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:35.700234    2577 patch_prober.go:28] interesting pod/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 09 13:06:35.700691 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:35.700300    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 09 13:06:36.704436 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:36.704404    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:36.704856 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:36.704666    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:36.705564 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:36.705551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:06:54.992748 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:54.992713    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"]
Sep 09 13:06:54.994692 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:54.994607    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7" podUID="ff1ef742-dba5-40cb-9445-4d872f4766e3" containerName="maas-api" containerID="cri-o://a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d" gracePeriod=30
Sep 09 13:06:55.044544 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.044510    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"]
Sep 09 13:06:55.044796 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.044771    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt" podUID="7cbd4ba1-22e8-4af5-8425-845e6eb2e980" containerName="payload-pre-processing" containerID="cri-o://9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31" gracePeriod=30
Sep 09 13:06:55.307686 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.307190    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:55.337492 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.337465    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:55.398411 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.398376    2577 generic.go:356] "Generic (PLEG): container finished" podID="ff1ef742-dba5-40cb-9445-4d872f4766e3" containerID="a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d" exitCode=0
Sep 09 13:06:55.398585 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.398421    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"
Sep 09 13:06:55.398585 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.398453    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7" event={"ID":"ff1ef742-dba5-40cb-9445-4d872f4766e3","Type":"ContainerDied","Data":"a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d"}
Sep 09 13:06:55.398585 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.398518    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7" event={"ID":"ff1ef742-dba5-40cb-9445-4d872f4766e3","Type":"ContainerDied","Data":"1afd25072891a4e3fc54e8036d85958e81093280ee8524207162f117b30b255c"}
Sep 09 13:06:55.398585 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.398551    2577 scope.go:117] "RemoveContainer" containerID="a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d"
Sep 09 13:06:55.399977 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.399950    2577 generic.go:356] "Generic (PLEG): container finished" podID="7cbd4ba1-22e8-4af5-8425-845e6eb2e980" containerID="9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31" exitCode=0
Sep 09 13:06:55.400097 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.399998    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"
Sep 09 13:06:55.400097 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.400032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt" event={"ID":"7cbd4ba1-22e8-4af5-8425-845e6eb2e980","Type":"ContainerDied","Data":"9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31"}
Sep 09 13:06:55.400097 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.400076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt" event={"ID":"7cbd4ba1-22e8-4af5-8425-845e6eb2e980","Type":"ContainerDied","Data":"862fd33003f4a950cd549260a5704476cf8c0a9f51295a645dca64249f595c6e"}
Sep 09 13:06:55.407543 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.407527    2577 scope.go:117] "RemoveContainer" containerID="a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d"
Sep 09 13:06:55.407832 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:06:55.407813    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d\": container with ID starting with a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d not found: ID does not exist" containerID="a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d"
Sep 09 13:06:55.407885 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.407840    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d"} err="failed to get container status \"a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d\": rpc error: code = NotFound desc = could not find container \"a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d\": container with ID starting with a4f092d4773e598e5a1a5578afc2729c0b52d7fb64911b96a6575c54b045596d not found: ID does not exist"
Sep 09 13:06:55.407885 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.407857    2577 scope.go:117] "RemoveContainer" containerID="9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31"
Sep 09 13:06:55.415747 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.415700    2577 scope.go:117] "RemoveContainer" containerID="9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31"
Sep 09 13:06:55.416067 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:06:55.416038    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31\": container with ID starting with 9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31 not found: ID does not exist" containerID="9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31"
Sep 09 13:06:55.416156 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.416073    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31"} err="failed to get container status \"9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31\": rpc error: code = NotFound desc = could not find container \"9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31\": container with ID starting with 9a11a25977dc7bbfcfd8304e3097aabacf6982a6d0121d94f1acca600808eb31 not found: ID does not exist"
Sep 09 13:06:55.431733 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.431713    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-maas-api-tls\") pod \"ff1ef742-dba5-40cb-9445-4d872f4766e3\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") "
Sep 09 13:06:55.431838 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.431751    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvsmk\" (UniqueName: \"kubernetes.io/projected/ff1ef742-dba5-40cb-9445-4d872f4766e3-kube-api-access-hvsmk\") pod \"ff1ef742-dba5-40cb-9445-4d872f4766e3\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") "
Sep 09 13:06:55.431838 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.431782    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-metrics-certs\") pod \"ff1ef742-dba5-40cb-9445-4d872f4766e3\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") "
Sep 09 13:06:55.431838 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.431824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ef742-dba5-40cb-9445-4d872f4766e3-openshift-service-ca\") pod \"ff1ef742-dba5-40cb-9445-4d872f4766e3\" (UID: \"ff1ef742-dba5-40cb-9445-4d872f4766e3\") "
Sep 09 13:06:55.432231 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.432202    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff1ef742-dba5-40cb-9445-4d872f4766e3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ff1ef742-dba5-40cb-9445-4d872f4766e3" (UID: "ff1ef742-dba5-40cb-9445-4d872f4766e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:06:55.434032 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.434009    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff1ef742-dba5-40cb-9445-4d872f4766e3-kube-api-access-hvsmk" (OuterVolumeSpecName: "kube-api-access-hvsmk") pod "ff1ef742-dba5-40cb-9445-4d872f4766e3" (UID: "ff1ef742-dba5-40cb-9445-4d872f4766e3"). InnerVolumeSpecName "kube-api-access-hvsmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:06:55.434111 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.434028    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ff1ef742-dba5-40cb-9445-4d872f4766e3" (UID: "ff1ef742-dba5-40cb-9445-4d872f4766e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:06:55.434111 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.434044    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ff1ef742-dba5-40cb-9445-4d872f4766e3" (UID: "ff1ef742-dba5-40cb-9445-4d872f4766e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:06:55.532998 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.532933    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-plugins-config-volume\") pod \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\" (UID: \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\") "
Sep 09 13:06:55.532998 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.532995    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzwt8\" (UniqueName: \"kubernetes.io/projected/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-kube-api-access-rzwt8\") pod \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\" (UID: \"7cbd4ba1-22e8-4af5-8425-845e6eb2e980\") "
Sep 09 13:06:55.533311 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.533294    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:06:55.533378 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.533317    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvsmk\" (UniqueName: \"kubernetes.io/projected/ff1ef742-dba5-40cb-9445-4d872f4766e3-kube-api-access-hvsmk\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:06:55.533378 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.533332    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff1ef742-dba5-40cb-9445-4d872f4766e3-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:06:55.533378 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.533346    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff1ef742-dba5-40cb-9445-4d872f4766e3-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:06:55.533378 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.533354    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7cbd4ba1-22e8-4af5-8425-845e6eb2e980" (UID: "7cbd4ba1-22e8-4af5-8425-845e6eb2e980"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:06:55.535272 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.535254    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-kube-api-access-rzwt8" (OuterVolumeSpecName: "kube-api-access-rzwt8") pod "7cbd4ba1-22e8-4af5-8425-845e6eb2e980" (UID: "7cbd4ba1-22e8-4af5-8425-845e6eb2e980"). InnerVolumeSpecName "kube-api-access-rzwt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:06:55.633969 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.633917    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:06:55.633969 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.633964    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzwt8\" (UniqueName: \"kubernetes.io/projected/7cbd4ba1-22e8-4af5-8425-845e6eb2e980-kube-api-access-rzwt8\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:06:55.723482 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.723448    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"]
Sep 09 13:06:55.727522 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.727497    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c2e681-7c86668d9b-7src7"]
Sep 09 13:06:55.736888 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.736860    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"]
Sep 09 13:06:55.740393 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:55.740369    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-c2e681-566fc65f94-vqtpt"]
Sep 09 13:06:56.515961 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:56.515926    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cbd4ba1-22e8-4af5-8425-845e6eb2e980" path="/var/lib/kubelet/pods/7cbd4ba1-22e8-4af5-8425-845e6eb2e980/volumes"
Sep 09 13:06:56.516386 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:06:56.516319    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff1ef742-dba5-40cb-9445-4d872f4766e3" path="/var/lib/kubelet/pods/ff1ef742-dba5-40cb-9445-4d872f4766e3/volumes"
Sep 09 13:07:01.925600 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:01.925562    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"]
Sep 09 13:07:01.926081 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:01.925850    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy" containerID="cri-o://93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48" gracePeriod=30
Sep 09 13:07:06.707776 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:06.707738    2577 patch_prober.go:28] interesting pod/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 13:07:06.708186 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:06.707812    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 13:07:07.184864 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.184836    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:07:07.236134 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236091    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5132e023-3cd9-4846-b206-260284ffc53c-istio-podinfo\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236134 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236140    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-certs\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236373 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236156    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-envoy\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236373 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236209    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-socket\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236373 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236233    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-istio-token\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236373 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236281    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5132e023-3cd9-4846-b206-260284ffc53c-istiod-ca-cert\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236374    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:07.236588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236424    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zxpm\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-kube-api-access-8zxpm\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236475    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-data\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-credential-socket\") pod \"5132e023-3cd9-4846-b206-260284ffc53c\" (UID: \"5132e023-3cd9-4846-b206-260284ffc53c\") "
Sep 09 13:07:07.236588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236570    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:07.236893 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236685    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5132e023-3cd9-4846-b206-260284ffc53c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:07.236954 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236892    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:07.237027 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.236994    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-data" (OuterVolumeSpecName: "istio-data") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:07.237260 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.237170    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.237260 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.237193    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5132e023-3cd9-4846-b206-260284ffc53c-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.237260 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.237208    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.237260 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.237220    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.237260 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.237234    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.238684 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.238603    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5132e023-3cd9-4846-b206-260284ffc53c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:07:07.238794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.238719    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:07.238794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.238774    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-kube-api-access-8zxpm" (OuterVolumeSpecName: "kube-api-access-8zxpm") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "kube-api-access-8zxpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:07.238878 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.238801    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-istio-token" (OuterVolumeSpecName: "istio-token") pod "5132e023-3cd9-4846-b206-260284ffc53c" (UID: "5132e023-3cd9-4846-b206-260284ffc53c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:07.337734 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.337594    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zxpm\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-kube-api-access-8zxpm\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.337734 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.337668    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5132e023-3cd9-4846-b206-260284ffc53c-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.337734 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.337679    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5132e023-3cd9-4846-b206-260284ffc53c-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.337734 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.337688    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5132e023-3cd9-4846-b206-260284ffc53c-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:07.468566 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.468533    2577 generic.go:356] "Generic (PLEG): container finished" podID="5132e023-3cd9-4846-b206-260284ffc53c" containerID="93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48" exitCode=0
Sep 09 13:07:07.468566 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.468573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" event={"ID":"5132e023-3cd9-4846-b206-260284ffc53c","Type":"ContainerDied","Data":"93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48"}
Sep 09 13:07:07.468806 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.468582    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"
Sep 09 13:07:07.468806 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.468617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw" event={"ID":"5132e023-3cd9-4846-b206-260284ffc53c","Type":"ContainerDied","Data":"d5acae3a553dfdfde3db03942e9c663f08fedd49bf27e19496fc909fbcd5d028"}
Sep 09 13:07:07.468806 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.468674    2577 scope.go:117] "RemoveContainer" containerID="93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48"
Sep 09 13:07:07.477993 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.477973    2577 scope.go:117] "RemoveContainer" containerID="404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33"
Sep 09 13:07:07.486489 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.486467    2577 scope.go:117] "RemoveContainer" containerID="93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48"
Sep 09 13:07:07.486858 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:07.486835    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48\": container with ID starting with 93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48 not found: ID does not exist" containerID="93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48"
Sep 09 13:07:07.486950 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.486867    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48"} err="failed to get container status \"93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48\": rpc error: code = NotFound desc = could not find container \"93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48\": container with ID starting with 93220bdda0064a42068bff3e02d3291f8dc6f3ee534b5417543f68a3d9f0dc48 not found: ID does not exist"
Sep 09 13:07:07.486950 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.486887    2577 scope.go:117] "RemoveContainer" containerID="404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33"
Sep 09 13:07:07.487182 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:07.487165    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33\": container with ID starting with 404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33 not found: ID does not exist" containerID="404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33"
Sep 09 13:07:07.487230 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.487188    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33"} err="failed to get container status \"404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33\": rpc error: code = NotFound desc = could not find container \"404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33\": container with ID starting with 404f5433d015a70d3758fc6bc92f50dde3a846184182a405c01164b2bf59bd33 not found: ID does not exist"
Sep 09 13:07:07.491556 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.491532    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"]
Sep 09 13:07:07.494832 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:07.494809    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-c2e681-openshift-default-679d984f48-g8jdw"]
Sep 09 13:07:08.515614 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:08.515584    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5132e023-3cd9-4846-b206-260284ffc53c" path="/var/lib/kubelet/pods/5132e023-3cd9-4846-b206-260284ffc53c/volumes"
Sep 09 13:07:12.311626 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.311591    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"]
Sep 09 13:07:12.312054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.311950    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy"
Sep 09 13:07:12.312054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.311962    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy"
Sep 09 13:07:12.312054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.311971    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cbd4ba1-22e8-4af5-8425-845e6eb2e980" containerName="payload-pre-processing"
Sep 09 13:07:12.312054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.311977    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cbd4ba1-22e8-4af5-8425-845e6eb2e980" containerName="payload-pre-processing"
Sep 09 13:07:12.312054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.311990    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff1ef742-dba5-40cb-9445-4d872f4766e3" containerName="maas-api"
Sep 09 13:07:12.312054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.311996    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff1ef742-dba5-40cb-9445-4d872f4766e3" containerName="maas-api"
Sep 09 13:07:12.312054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.312048    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy"
Sep 09 13:07:12.312054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.312056    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cbd4ba1-22e8-4af5-8425-845e6eb2e980" containerName="payload-pre-processing"
Sep 09 13:07:12.312339 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.312066    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff1ef742-dba5-40cb-9445-4d872f4766e3" containerName="maas-api"
Sep 09 13:07:12.316227 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.316204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.318630 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.318602    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-73011135-openshift-default-dockercfg-t498m\""
Sep 09 13:07:12.326769 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.326740    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"]
Sep 09 13:07:12.384434 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384394    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-podinfo\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.384434 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384434    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-certs\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.384693 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-token\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.384693 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-credential-socket\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.384693 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fad17ac-5b46-47ac-bed4-80ea534aa377-istiod-ca-cert\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.384693 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-envoy\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.384693 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c62mv\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-kube-api-access-c62mv\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.384693 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-socket\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.384907 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.384728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-data\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.485837 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.485796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-credential-socket\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.485837 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.485840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fad17ac-5b46-47ac-bed4-80ea534aa377-istiod-ca-cert\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486101 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.485878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-envoy\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486101 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.485901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c62mv\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-kube-api-access-c62mv\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486101 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.485958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-socket\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486251 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-data\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486251 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-podinfo\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486251 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-certs\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486251 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-token\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486456 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-socket\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486562    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-data\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486756 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486611    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-certs\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486756 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486673    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fad17ac-5b46-47ac-bed4-80ea534aa377-istiod-ca-cert\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.486756 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.486670    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-credential-socket\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.488478 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.488453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-envoy\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.488790 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.488770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-podinfo\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.494521 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.494488    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c62mv\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-kube-api-access-c62mv\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.494621 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.494487    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-token\") pod \"e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") " pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.628162 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.628128    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:12.905691 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.905661    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"]
Sep 09 13:07:12.908378 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:07:12.908348    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fad17ac_5b46_47ac_bed4_80ea534aa377.slice/crio-a488a314b6bcf48675cb98c0e16fc72774d8c5b3ee277467fe146a5bbca451d0 WatchSource:0}: Error finding container a488a314b6bcf48675cb98c0e16fc72774d8c5b3ee277467fe146a5bbca451d0: Status 404 returned error can't find the container with id a488a314b6bcf48675cb98c0e16fc72774d8c5b3ee277467fe146a5bbca451d0
Sep 09 13:07:12.910661 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.910610    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:07:12.910757 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.910734    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:07:12.910827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:12.910780    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:07:13.499159 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:13.499119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" event={"ID":"0fad17ac-5b46-47ac-bed4-80ea534aa377","Type":"ContainerStarted","Data":"680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae"}
Sep 09 13:07:13.499563 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:13.499173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" event={"ID":"0fad17ac-5b46-47ac-bed4-80ea534aa377","Type":"ContainerStarted","Data":"a488a314b6bcf48675cb98c0e16fc72774d8c5b3ee277467fe146a5bbca451d0"}
Sep 09 13:07:13.517799 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:13.517751    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" podStartSLOduration=1.517736244 podStartE2EDuration="1.517736244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:07:13.51601074 +0000 UTC m=+1119.516676855" watchObservedRunningTime="2026-09-09 13:07:13.517736244 +0000 UTC m=+1119.518402344"
Sep 09 13:07:13.628902 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:13.628869    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:13.638440 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:13.638408    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:14.503571 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:14.503544    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:14.504372 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:14.504354    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:17.165201 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.165167    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"]
Sep 09 13:07:17.165698 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.165678    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy"
Sep 09 13:07:17.165698 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.165701    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy"
Sep 09 13:07:17.165849 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.165814    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5132e023-3cd9-4846-b206-260284ffc53c" containerName="istio-proxy"
Sep 09 13:07:17.169053 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.169029    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:17.171397 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.171374    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-73011135\""
Sep 09 13:07:17.171508 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.171465    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-73011135-dockercfg-682sn\""
Sep 09 13:07:17.180947 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.180926    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"]
Sep 09 13:07:17.202830 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.202808    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"]
Sep 09 13:07:17.205846 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.205832    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:17.214686 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.214664    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"]
Sep 09 13:07:17.230891 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.230868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp\" (UID: \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:17.230991 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.230976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c89rd\" (UniqueName: \"kubernetes.io/projected/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-kube-api-access-c89rd\") pod \"payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp\" (UID: \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:17.331628 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.331593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-plugins-config-volume\") pod \"payload-processing-e2e-mt-73011135-588566f8cc-rhd97\" (UID: \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\") " pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:17.331846 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.331665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbht2\" (UniqueName: \"kubernetes.io/projected/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-kube-api-access-mbht2\") pod \"payload-processing-e2e-mt-73011135-588566f8cc-rhd97\" (UID: \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\") " pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:17.331846 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.331727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c89rd\" (UniqueName: \"kubernetes.io/projected/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-kube-api-access-c89rd\") pod \"payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp\" (UID: \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:17.331846 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.331791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp\" (UID: \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:17.332481 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.332459    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp\" (UID: \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:17.339276 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.339255    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c89rd\" (UniqueName: \"kubernetes.io/projected/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-kube-api-access-c89rd\") pod \"payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp\" (UID: \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:17.432417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.432354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-plugins-config-volume\") pod \"payload-processing-e2e-mt-73011135-588566f8cc-rhd97\" (UID: \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\") " pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:17.432417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.432387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbht2\" (UniqueName: \"kubernetes.io/projected/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-kube-api-access-mbht2\") pod \"payload-processing-e2e-mt-73011135-588566f8cc-rhd97\" (UID: \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\") " pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:17.433060 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.433042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-plugins-config-volume\") pod \"payload-processing-e2e-mt-73011135-588566f8cc-rhd97\" (UID: \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\") " pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:17.440602 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.440583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbht2\" (UniqueName: \"kubernetes.io/projected/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-kube-api-access-mbht2\") pod \"payload-processing-e2e-mt-73011135-588566f8cc-rhd97\" (UID: \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\") " pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:17.481482 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.481459    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:17.515996 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.515947    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:17.754476 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.754450    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"]
Sep 09 13:07:17.756283 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:07:17.756255    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8caa9ac8_71a8_4f31_bdf7_cdeebe755576.slice/crio-bfcc9110fbe132d4acd2ea7ae5a1c985c8497bf42f37f30a155f74a5d86e3ba9 WatchSource:0}: Error finding container bfcc9110fbe132d4acd2ea7ae5a1c985c8497bf42f37f30a155f74a5d86e3ba9: Status 404 returned error can't find the container with id bfcc9110fbe132d4acd2ea7ae5a1c985c8497bf42f37f30a155f74a5d86e3ba9
Sep 09 13:07:17.795437 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:17.795411    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"]
Sep 09 13:07:17.796669 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:07:17.796629    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d65ac31_43c6_4ddf_8ed7_8c38a4e5a563.slice/crio-7e10d1f477e99387afb7360dfd3378ee5e9e4be91c13c5db22b651b9078e89e5 WatchSource:0}: Error finding container 7e10d1f477e99387afb7360dfd3378ee5e9e4be91c13c5db22b651b9078e89e5: Status 404 returned error can't find the container with id 7e10d1f477e99387afb7360dfd3378ee5e9e4be91c13c5db22b651b9078e89e5
Sep 09 13:07:18.523792 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:18.523757    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp" event={"ID":"8caa9ac8-71a8-4f31-bdf7-cdeebe755576","Type":"ContainerStarted","Data":"d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40"}
Sep 09 13:07:18.524222 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:18.523817    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp" event={"ID":"8caa9ac8-71a8-4f31-bdf7-cdeebe755576","Type":"ContainerStarted","Data":"bfcc9110fbe132d4acd2ea7ae5a1c985c8497bf42f37f30a155f74a5d86e3ba9"}
Sep 09 13:07:18.524222 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:18.523936    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:18.525255 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:18.525234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97" event={"ID":"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563","Type":"ContainerStarted","Data":"6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22"}
Sep 09 13:07:18.525340 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:18.525277    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97" event={"ID":"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563","Type":"ContainerStarted","Data":"7e10d1f477e99387afb7360dfd3378ee5e9e4be91c13c5db22b651b9078e89e5"}
Sep 09 13:07:18.525390 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:18.525338    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:18.539480 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:18.539428    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp" podStartSLOduration=1.5394140570000001 podStartE2EDuration="1.539414057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:07:18.537574597 +0000 UTC m=+1124.538240710" watchObservedRunningTime="2026-09-09 13:07:18.539414057 +0000 UTC m=+1124.540080177"
Sep 09 13:07:18.553006 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:18.552956    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97" podStartSLOduration=1.552941663 podStartE2EDuration="1.552941663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:07:18.551778588 +0000 UTC m=+1124.552444689" watchObservedRunningTime="2026-09-09 13:07:18.552941663 +0000 UTC m=+1124.553607805"
Sep 09 13:07:29.532553 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:29.532515    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:29.533140 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:29.532613    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:29.547212 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:29.547176    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"]
Sep 09 13:07:36.628859 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:36.628830    2577 generic.go:356] "Generic (PLEG): container finished" podID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerID="680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae" exitCode=0
Sep 09 13:07:36.629303 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:36.628889    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" event={"ID":"0fad17ac-5b46-47ac-bed4-80ea534aa377","Type":"ContainerDied","Data":"680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae"}
Sep 09 13:07:36.629303 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:36.629261    2577 scope.go:117] "RemoveContainer" containerID="680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae"
Sep 09 13:07:36.630321 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:36.630295    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:07:36.630417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:36.630359    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:07:36.630417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:36.630389    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:07:37.635086 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:37.635048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" event={"ID":"0fad17ac-5b46-47ac-bed4-80ea534aa377","Type":"ContainerStarted","Data":"d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5"}
Sep 09 13:07:38.628478 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:38.628443    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:38.628690 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:38.628672    2577 patch_prober.go:28] interesting pod/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 13:07:38.628744 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:38.628720    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 13:07:39.629209 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:39.629176    2577 patch_prober.go:28] interesting pod/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 13:07:39.629586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:39.629240    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 13:07:40.629504 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:40.629470    2577 patch_prober.go:28] interesting pod/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 13:07:40.629976 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:40.629534    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 13:07:41.628688 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:41.628618    2577 patch_prober.go:28] interesting pod/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 13:07:41.628900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:41.628715    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 13:07:41.716293 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:41.716256    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"]
Sep 09 13:07:41.716795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:41.716499    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2" podUID="f903cc10-1abf-4f0f-a05b-4d30e5e0c086" containerName="maas-api" containerID="cri-o://a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981" gracePeriod=30
Sep 09 13:07:41.772058 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:41.769375    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"]
Sep 09 13:07:41.791165 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:41.791126    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"]
Sep 09 13:07:41.799732 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:41.791986    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r" podUID="c233dc28-13ec-4836-8532-f0c8e48fe1bc" containerName="payload-processing" containerID="cri-o://585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e" gracePeriod=30
Sep 09 13:07:41.800016 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:41.791615    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w" podUID="c618af0b-148f-4c24-9e05-37c26b9fb268" containerName="payload-pre-processing" containerID="cri-o://f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1" gracePeriod=30
Sep 09 13:07:42.130729 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.130707    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:07:42.135452 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.135365    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:07:42.257454 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.257424    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:07:42.263245 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.263213    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c233dc28-13ec-4836-8532-f0c8e48fe1bc-plugins-config-volume\") pod \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\" (UID: \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\") "
Sep 09 13:07:42.263386 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.263307    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6wxx\" (UniqueName: \"kubernetes.io/projected/c618af0b-148f-4c24-9e05-37c26b9fb268-kube-api-access-j6wxx\") pod \"c618af0b-148f-4c24-9e05-37c26b9fb268\" (UID: \"c618af0b-148f-4c24-9e05-37c26b9fb268\") "
Sep 09 13:07:42.263386 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.263333    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c618af0b-148f-4c24-9e05-37c26b9fb268-plugins-config-volume\") pod \"c618af0b-148f-4c24-9e05-37c26b9fb268\" (UID: \"c618af0b-148f-4c24-9e05-37c26b9fb268\") "
Sep 09 13:07:42.263386 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.263374    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg69x\" (UniqueName: \"kubernetes.io/projected/c233dc28-13ec-4836-8532-f0c8e48fe1bc-kube-api-access-cg69x\") pod \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\" (UID: \"c233dc28-13ec-4836-8532-f0c8e48fe1bc\") "
Sep 09 13:07:42.263665 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.263619    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c233dc28-13ec-4836-8532-f0c8e48fe1bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c233dc28-13ec-4836-8532-f0c8e48fe1bc" (UID: "c233dc28-13ec-4836-8532-f0c8e48fe1bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:42.263750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.263725    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c233dc28-13ec-4836-8532-f0c8e48fe1bc-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:42.263814 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.263786    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c618af0b-148f-4c24-9e05-37c26b9fb268-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c618af0b-148f-4c24-9e05-37c26b9fb268" (UID: "c618af0b-148f-4c24-9e05-37c26b9fb268"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:42.265613 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.265584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c618af0b-148f-4c24-9e05-37c26b9fb268-kube-api-access-j6wxx" (OuterVolumeSpecName: "kube-api-access-j6wxx") pod "c618af0b-148f-4c24-9e05-37c26b9fb268" (UID: "c618af0b-148f-4c24-9e05-37c26b9fb268"). InnerVolumeSpecName "kube-api-access-j6wxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:42.265764 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.265608    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c233dc28-13ec-4836-8532-f0c8e48fe1bc-kube-api-access-cg69x" (OuterVolumeSpecName: "kube-api-access-cg69x") pod "c233dc28-13ec-4836-8532-f0c8e48fe1bc" (UID: "c233dc28-13ec-4836-8532-f0c8e48fe1bc"). InnerVolumeSpecName "kube-api-access-cg69x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:42.364625 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.364590    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-metrics-certs\") pod \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") "
Sep 09 13:07:42.364828 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.364723    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-openshift-service-ca\") pod \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") "
Sep 09 13:07:42.364828 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.364774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-maas-api-tls\") pod \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") "
Sep 09 13:07:42.364828 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.364798    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drlhl\" (UniqueName: \"kubernetes.io/projected/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-kube-api-access-drlhl\") pod \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\" (UID: \"f903cc10-1abf-4f0f-a05b-4d30e5e0c086\") "
Sep 09 13:07:42.365030 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.365012    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6wxx\" (UniqueName: \"kubernetes.io/projected/c618af0b-148f-4c24-9e05-37c26b9fb268-kube-api-access-j6wxx\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:42.365098 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.365040    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c618af0b-148f-4c24-9e05-37c26b9fb268-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:42.365098 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.365055    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg69x\" (UniqueName: \"kubernetes.io/projected/c233dc28-13ec-4836-8532-f0c8e48fe1bc-kube-api-access-cg69x\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:42.365098 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.365054    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f903cc10-1abf-4f0f-a05b-4d30e5e0c086" (UID: "f903cc10-1abf-4f0f-a05b-4d30e5e0c086"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:42.372840 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.372738    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f903cc10-1abf-4f0f-a05b-4d30e5e0c086" (UID: "f903cc10-1abf-4f0f-a05b-4d30e5e0c086"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:07:42.374865 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.374753    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f903cc10-1abf-4f0f-a05b-4d30e5e0c086" (UID: "f903cc10-1abf-4f0f-a05b-4d30e5e0c086"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:07:42.374865 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.374837    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-kube-api-access-drlhl" (OuterVolumeSpecName: "kube-api-access-drlhl") pod "f903cc10-1abf-4f0f-a05b-4d30e5e0c086" (UID: "f903cc10-1abf-4f0f-a05b-4d30e5e0c086"). InnerVolumeSpecName "kube-api-access-drlhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:42.466130 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.466080    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:42.466130 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.466126    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:42.466347 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.466142    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drlhl\" (UniqueName: \"kubernetes.io/projected/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-kube-api-access-drlhl\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:42.466347 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.466156    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f903cc10-1abf-4f0f-a05b-4d30e5e0c086-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:42.629580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.629533    2577 patch_prober.go:28] interesting pod/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 09 13:07:42.629945 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.629881    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 09 13:07:42.666400 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.666352    2577 generic.go:356] "Generic (PLEG): container finished" podID="c233dc28-13ec-4836-8532-f0c8e48fe1bc" containerID="585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e" exitCode=0
Sep 09 13:07:42.666607 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.666421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r" event={"ID":"c233dc28-13ec-4836-8532-f0c8e48fe1bc","Type":"ContainerDied","Data":"585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e"}
Sep 09 13:07:42.666607 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.666443    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"
Sep 09 13:07:42.666607 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.666538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r" event={"ID":"c233dc28-13ec-4836-8532-f0c8e48fe1bc","Type":"ContainerDied","Data":"581f2cda907d9388caa7d73895dc3bb62cd8298fb96bd61b04c816579ff8fa61"}
Sep 09 13:07:42.666607 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.666593    2577 scope.go:117] "RemoveContainer" containerID="585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e"
Sep 09 13:07:42.669752 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.669718    2577 generic.go:356] "Generic (PLEG): container finished" podID="f903cc10-1abf-4f0f-a05b-4d30e5e0c086" containerID="a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981" exitCode=0
Sep 09 13:07:42.669862 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.669777    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"
Sep 09 13:07:42.669929 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.669903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2" event={"ID":"f903cc10-1abf-4f0f-a05b-4d30e5e0c086","Type":"ContainerDied","Data":"a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981"}
Sep 09 13:07:42.669989 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.669942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2" event={"ID":"f903cc10-1abf-4f0f-a05b-4d30e5e0c086","Type":"ContainerDied","Data":"6666baf725ef06977547cd7d2a30292a25f8a9a136d56204b77557df2338d050"}
Sep 09 13:07:42.671955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.671928    2577 generic.go:356] "Generic (PLEG): container finished" podID="c618af0b-148f-4c24-9e05-37c26b9fb268" containerID="f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1" exitCode=0
Sep 09 13:07:42.672081 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.672040    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"
Sep 09 13:07:42.672564 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.672540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w" event={"ID":"c618af0b-148f-4c24-9e05-37c26b9fb268","Type":"ContainerDied","Data":"f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1"}
Sep 09 13:07:42.672687 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.672591    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w" event={"ID":"c618af0b-148f-4c24-9e05-37c26b9fb268","Type":"ContainerDied","Data":"c41ce7991bf902554b5fdb38d9bb2dbd47e2ada1117692109cbce6645ffb450f"}
Sep 09 13:07:42.681658 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.681618    2577 scope.go:117] "RemoveContainer" containerID="585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e"
Sep 09 13:07:42.682104 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:42.682066    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e\": container with ID starting with 585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e not found: ID does not exist" containerID="585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e"
Sep 09 13:07:42.682249 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.682224    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e"} err="failed to get container status \"585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e\": rpc error: code = NotFound desc = could not find container \"585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e\": container with ID starting with 585328f3721ac6fbdb4a4e57ad47d87088bb5f4d7da3e8bd5b83284a47efef5e not found: ID does not exist"
Sep 09 13:07:42.682371 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.682349    2577 scope.go:117] "RemoveContainer" containerID="a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981"
Sep 09 13:07:42.689210 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.689184    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"]
Sep 09 13:07:42.693285 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.693262    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3b4b96-6d7c7d949-7w77r"]
Sep 09 13:07:42.695365 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.695345    2577 scope.go:117] "RemoveContainer" containerID="a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981"
Sep 09 13:07:42.695848 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:42.695809    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981\": container with ID starting with a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981 not found: ID does not exist" containerID="a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981"
Sep 09 13:07:42.695969 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.695844    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981"} err="failed to get container status \"a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981\": rpc error: code = NotFound desc = could not find container \"a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981\": container with ID starting with a7c985cf070f460afe8e4775b24d45cd1e4c12ffe6250f6e59244686cbedf981 not found: ID does not exist"
Sep 09 13:07:42.695969 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.695868    2577 scope.go:117] "RemoveContainer" containerID="f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1"
Sep 09 13:07:42.705946 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.705902    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"]
Sep 09 13:07:42.707712 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.707611    2577 scope.go:117] "RemoveContainer" containerID="f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1"
Sep 09 13:07:42.708389 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:42.708300    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1\": container with ID starting with f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1 not found: ID does not exist" containerID="f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1"
Sep 09 13:07:42.708389 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.708336    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1"} err="failed to get container status \"f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1\": rpc error: code = NotFound desc = could not find container \"f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1\": container with ID starting with f954734004bf646e82b1e51569c2a9b9149483ed0a680c4585e7354d76eb12c1 not found: ID does not exist"
Sep 09 13:07:42.711453 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.711428    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3b4b96-5bb4cf944b-nzqs2"]
Sep 09 13:07:42.722714 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.722611    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"]
Sep 09 13:07:42.725525 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:42.725499    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3b4b96-d5fbf89c8-pbf7w"]
Sep 09 13:07:43.633248 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:43.633214    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:43.633544 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:43.633515    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:43.634108 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:43.634092    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:07:44.514840 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.514810    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c233dc28-13ec-4836-8532-f0c8e48fe1bc" path="/var/lib/kubelet/pods/c233dc28-13ec-4836-8532-f0c8e48fe1bc/volumes"
Sep 09 13:07:44.515200 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.515125    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c618af0b-148f-4c24-9e05-37c26b9fb268" path="/var/lib/kubelet/pods/c618af0b-148f-4c24-9e05-37c26b9fb268/volumes"
Sep 09 13:07:44.515452 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.515439    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f903cc10-1abf-4f0f-a05b-4d30e5e0c086" path="/var/lib/kubelet/pods/f903cc10-1abf-4f0f-a05b-4d30e5e0c086/volumes"
Sep 09 13:07:44.600950 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.600919    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f" podUID="04b1e7be-4656-42aa-a6a3-cfd591a32f12" containerName="main" containerID="cri-o://036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424" gracePeriod=45
Sep 09 13:07:44.841949 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.841923    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:07:44.987490 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.987462    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nz62v\" (UniqueName: \"kubernetes.io/projected/04b1e7be-4656-42aa-a6a3-cfd591a32f12-kube-api-access-nz62v\") pod \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") "
Sep 09 13:07:44.987667 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.987511    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tmp-dir\") pod \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") "
Sep 09 13:07:44.987667 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.987552    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tls-certs\") pod \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") "
Sep 09 13:07:44.987667 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.987582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-model-cache\") pod \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") "
Sep 09 13:07:44.987667 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.987621    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-dshm\") pod \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") "
Sep 09 13:07:44.987894 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.987777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-home\") pod \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\" (UID: \"04b1e7be-4656-42aa-a6a3-cfd591a32f12\") "
Sep 09 13:07:44.987960 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.987895    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "04b1e7be-4656-42aa-a6a3-cfd591a32f12" (UID: "04b1e7be-4656-42aa-a6a3-cfd591a32f12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:44.987960 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.987906    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-model-cache" (OuterVolumeSpecName: "model-cache") pod "04b1e7be-4656-42aa-a6a3-cfd591a32f12" (UID: "04b1e7be-4656-42aa-a6a3-cfd591a32f12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:44.988054 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.988018    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-home" (OuterVolumeSpecName: "home") pod "04b1e7be-4656-42aa-a6a3-cfd591a32f12" (UID: "04b1e7be-4656-42aa-a6a3-cfd591a32f12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:44.988110 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.988076    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-home\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:44.988110 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.988090    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tmp-dir\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:44.988110 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.988099    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-model-cache\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:44.989915 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.989892    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b1e7be-4656-42aa-a6a3-cfd591a32f12-kube-api-access-nz62v" (OuterVolumeSpecName: "kube-api-access-nz62v") pod "04b1e7be-4656-42aa-a6a3-cfd591a32f12" (UID: "04b1e7be-4656-42aa-a6a3-cfd591a32f12"). InnerVolumeSpecName "kube-api-access-nz62v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:44.990004 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.989916    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-dshm" (OuterVolumeSpecName: "dshm") pod "04b1e7be-4656-42aa-a6a3-cfd591a32f12" (UID: "04b1e7be-4656-42aa-a6a3-cfd591a32f12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:44.990422 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:44.990396    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04b1e7be-4656-42aa-a6a3-cfd591a32f12" (UID: "04b1e7be-4656-42aa-a6a3-cfd591a32f12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:07:45.089102 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.089041    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b1e7be-4656-42aa-a6a3-cfd591a32f12-dshm\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:45.089102 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.089064    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz62v\" (UniqueName: \"kubernetes.io/projected/04b1e7be-4656-42aa-a6a3-cfd591a32f12-kube-api-access-nz62v\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:45.089102 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.089080    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b1e7be-4656-42aa-a6a3-cfd591a32f12-tls-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:45.694653 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.694598    2577 generic.go:356] "Generic (PLEG): container finished" podID="04b1e7be-4656-42aa-a6a3-cfd591a32f12" containerID="036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424" exitCode=0
Sep 09 13:07:45.695062 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.694677    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"
Sep 09 13:07:45.695062 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.694700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f" event={"ID":"04b1e7be-4656-42aa-a6a3-cfd591a32f12","Type":"ContainerDied","Data":"036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424"}
Sep 09 13:07:45.695062 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.694736    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f" event={"ID":"04b1e7be-4656-42aa-a6a3-cfd591a32f12","Type":"ContainerDied","Data":"e8c71b221553d72c5b642630479b736e926df0d8e80bafe6b56d0af196fec797"}
Sep 09 13:07:45.695062 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.694756    2577 scope.go:117] "RemoveContainer" containerID="036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424"
Sep 09 13:07:45.704681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.704627    2577 scope.go:117] "RemoveContainer" containerID="036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424"
Sep 09 13:07:45.704981 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:45.704956    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424\": container with ID starting with 036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424 not found: ID does not exist" containerID="036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424"
Sep 09 13:07:45.705059 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.704992    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424"} err="failed to get container status \"036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424\": rpc error: code = NotFound desc = could not find container \"036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424\": container with ID starting with 036d325f5d1c26eefa2ef49a29e7ae13975db4022ed617619bf5934f2d285424 not found: ID does not exist"
Sep 09 13:07:45.717037 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.717005    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"]
Sep 09 13:07:45.718694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:45.718673    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-3b4b96/test-model-3b4b96-kserve-6b5c4cb9df-jzp8f"]
Sep 09 13:07:46.515811 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:46.515782    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b1e7be-4656-42aa-a6a3-cfd591a32f12" path="/var/lib/kubelet/pods/04b1e7be-4656-42aa-a6a3-cfd591a32f12/volumes"
Sep 09 13:07:47.776900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.776869    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"]
Sep 09 13:07:47.777256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777172    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f903cc10-1abf-4f0f-a05b-4d30e5e0c086" containerName="maas-api"
Sep 09 13:07:47.777256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777184    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f903cc10-1abf-4f0f-a05b-4d30e5e0c086" containerName="maas-api"
Sep 09 13:07:47.777256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777192    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b1e7be-4656-42aa-a6a3-cfd591a32f12" containerName="main"
Sep 09 13:07:47.777256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777197    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b1e7be-4656-42aa-a6a3-cfd591a32f12" containerName="main"
Sep 09 13:07:47.777256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777216    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c233dc28-13ec-4836-8532-f0c8e48fe1bc" containerName="payload-processing"
Sep 09 13:07:47.777256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777222    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c233dc28-13ec-4836-8532-f0c8e48fe1bc" containerName="payload-processing"
Sep 09 13:07:47.777256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777229    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c618af0b-148f-4c24-9e05-37c26b9fb268" containerName="payload-pre-processing"
Sep 09 13:07:47.777256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777234    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c618af0b-148f-4c24-9e05-37c26b9fb268" containerName="payload-pre-processing"
Sep 09 13:07:47.777505 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777296    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c618af0b-148f-4c24-9e05-37c26b9fb268" containerName="payload-pre-processing"
Sep 09 13:07:47.777505 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777303    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c233dc28-13ec-4836-8532-f0c8e48fe1bc" containerName="payload-processing"
Sep 09 13:07:47.777505 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777312    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b1e7be-4656-42aa-a6a3-cfd591a32f12" containerName="main"
Sep 09 13:07:47.777505 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.777318    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f903cc10-1abf-4f0f-a05b-4d30e5e0c086" containerName="maas-api"
Sep 09 13:07:47.779750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.779735    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"
Sep 09 13:07:47.782577 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.782550    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:07:47.782845 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.782831    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 13:07:47.804235 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.804214    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"]
Sep 09 13:07:47.815498 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.815472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ee1f6d-d273-4d22-82f6-6b60da8cd438-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-73011135-gzt4z\" (UID: \"21ee1f6d-d273-4d22-82f6-6b60da8cd438\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"
Sep 09 13:07:47.916837 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.916802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ee1f6d-d273-4d22-82f6-6b60da8cd438-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-73011135-gzt4z\" (UID: \"21ee1f6d-d273-4d22-82f6-6b60da8cd438\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"
Sep 09 13:07:47.917405 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:47.917382    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ee1f6d-d273-4d22-82f6-6b60da8cd438-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-73011135-gzt4z\" (UID: \"21ee1f6d-d273-4d22-82f6-6b60da8cd438\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"
Sep 09 13:07:48.089805 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:48.089721    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"
Sep 09 13:07:48.359794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:48.359770    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"]
Sep 09 13:07:48.362049 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:07:48.362021    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21ee1f6d_d273_4d22_82f6_6b60da8cd438.slice/crio-da6cc25f9bf5482452a025fc4686c3d54f2696f64e3db340d5d5342562f0bb26 WatchSource:0}: Error finding container da6cc25f9bf5482452a025fc4686c3d54f2696f64e3db340d5d5342562f0bb26: Status 404 returned error can't find the container with id da6cc25f9bf5482452a025fc4686c3d54f2696f64e3db340d5d5342562f0bb26
Sep 09 13:07:48.710270 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:48.710240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z" event={"ID":"21ee1f6d-d273-4d22-82f6-6b60da8cd438","Type":"ContainerStarted","Data":"da6cc25f9bf5482452a025fc4686c3d54f2696f64e3db340d5d5342562f0bb26"}
Sep 09 13:07:49.039104 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.039016    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"]
Sep 09 13:07:49.039535 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.039216    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy" containerID="cri-o://4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c" gracePeriod=30
Sep 09 13:07:49.172284 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.172243    2577 patch_prober.go:28] interesting pod/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 13:07:49.172430 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.172313    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 13:07:49.404161 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.404129    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"]
Sep 09 13:07:49.407160 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.407145    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"
Sep 09 13:07:49.415206 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.415183    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"]
Sep 09 13:07:49.428558 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.428533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f6ba2e9-99ea-4a46-adfb-58976b159a99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5\" (UID: \"1f6ba2e9-99ea-4a46-adfb-58976b159a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"
Sep 09 13:07:49.529209 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.529181    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f6ba2e9-99ea-4a46-adfb-58976b159a99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5\" (UID: \"1f6ba2e9-99ea-4a46-adfb-58976b159a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"
Sep 09 13:07:49.529795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.529779    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f6ba2e9-99ea-4a46-adfb-58976b159a99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5\" (UID: \"1f6ba2e9-99ea-4a46-adfb-58976b159a99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"
Sep 09 13:07:49.718681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.718580    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"
Sep 09 13:07:49.989284 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:49.989263    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"]
Sep 09 13:07:49.991312 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:07:49.991286    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f6ba2e9_99ea_4a46_adfb_58976b159a99.slice/crio-acdfce60ab5004b44a0d153c00b584f3c53f23d88229a2c4305811fcfd8c80f7 WatchSource:0}: Error finding container acdfce60ab5004b44a0d153c00b584f3c53f23d88229a2c4305811fcfd8c80f7: Status 404 returned error can't find the container with id acdfce60ab5004b44a0d153c00b584f3c53f23d88229a2c4305811fcfd8c80f7
Sep 09 13:07:50.721856 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:50.721821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5" event={"ID":"1f6ba2e9-99ea-4a46-adfb-58976b159a99","Type":"ContainerStarted","Data":"acdfce60ab5004b44a0d153c00b584f3c53f23d88229a2c4305811fcfd8c80f7"}
Sep 09 13:07:51.727405 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:51.727372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z" event={"ID":"21ee1f6d-d273-4d22-82f6-6b60da8cd438","Type":"ContainerStarted","Data":"ef1b99f39e8ab91969d6455b687bb244d42d6e127e0e26cb7d4b172fb0692e23"}
Sep 09 13:07:51.742670 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:51.742563    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z" podStartSLOduration=1.609667851 podStartE2EDuration="4.742541893s" totalImagesPullingTime="3.132874042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:07:51.741988516 +0000 UTC m=+1157.742654619" watchObservedRunningTime="2026-09-09 13:07:51.742541893 +0000 UTC m=+1157.743207993"
Sep 09 13:07:52.733180 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:52.733144    2577 generic.go:356] "Generic (PLEG): container finished" podID="1f6ba2e9-99ea-4a46-adfb-58976b159a99" containerID="12b148d2288cc7aeb8a4e79654a15922ede954f96a21a0b9aafc2928262b3c53" exitCode=0
Sep 09 13:07:52.733627 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:52.733238    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5" event={"ID":"1f6ba2e9-99ea-4a46-adfb-58976b159a99","Type":"ContainerDied","Data":"12b148d2288cc7aeb8a4e79654a15922ede954f96a21a0b9aafc2928262b3c53"}
Sep 09 13:07:52.734627 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:52.734604    2577 generic.go:356] "Generic (PLEG): container finished" podID="21ee1f6d-d273-4d22-82f6-6b60da8cd438" containerID="ef1b99f39e8ab91969d6455b687bb244d42d6e127e0e26cb7d4b172fb0692e23" exitCode=0
Sep 09 13:07:52.734703 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:52.734670    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z" event={"ID":"21ee1f6d-d273-4d22-82f6-6b60da8cd438","Type":"ContainerDied","Data":"ef1b99f39e8ab91969d6455b687bb244d42d6e127e0e26cb7d4b172fb0692e23"}
Sep 09 13:07:53.894361 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:53.894333    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"
Sep 09 13:07:53.897306 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:53.897284    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"
Sep 09 13:07:53.968828 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:53.968791    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f6ba2e9-99ea-4a46-adfb-58976b159a99-maas-api-service-ca\") pod \"1f6ba2e9-99ea-4a46-adfb-58976b159a99\" (UID: \"1f6ba2e9-99ea-4a46-adfb-58976b159a99\") "
Sep 09 13:07:53.969001 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:53.968893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ee1f6d-d273-4d22-82f6-6b60da8cd438-maas-api-service-ca\") pod \"21ee1f6d-d273-4d22-82f6-6b60da8cd438\" (UID: \"21ee1f6d-d273-4d22-82f6-6b60da8cd438\") "
Sep 09 13:07:53.969245 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:53.969221    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21ee1f6d-d273-4d22-82f6-6b60da8cd438-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "21ee1f6d-d273-4d22-82f6-6b60da8cd438" (UID: "21ee1f6d-d273-4d22-82f6-6b60da8cd438"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:53.969281 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:53.969253    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f6ba2e9-99ea-4a46-adfb-58976b159a99-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f6ba2e9-99ea-4a46-adfb-58976b159a99" (UID: "1f6ba2e9-99ea-4a46-adfb-58976b159a99"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:54.070421 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.070384    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ee1f6d-d273-4d22-82f6-6b60da8cd438-maas-api-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.070421 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.070418    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f6ba2e9-99ea-4a46-adfb-58976b159a99-maas-api-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.292699 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.292672    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:07:54.372580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372487    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-certs\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372524    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-credential-socket\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-socket\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372567    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-podinfo\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372602    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-data\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372622    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-envoy\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372703    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbcbefac-4382-4a7a-93ea-9e5492576609-istiod-ca-cert\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372728    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-token\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372790    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7462m\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-kube-api-access-7462m\") pod \"cbcbefac-4382-4a7a-93ea-9e5492576609\" (UID: \"cbcbefac-4382-4a7a-93ea-9e5492576609\") "
Sep 09 13:07:54.372955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372836    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:54.372955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372858    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:54.372955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.372872    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:54.373295 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.373035    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-data" (OuterVolumeSpecName: "istio-data") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:54.373295 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.373082    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbcbefac-4382-4a7a-93ea-9e5492576609-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:54.373295 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.373159    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.373295 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.373176    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.373295 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.373190    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.373295 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.373201    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.375120 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.375092    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:54.375231 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.375137    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:07:54.375488 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.375464    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-kube-api-access-7462m" (OuterVolumeSpecName: "kube-api-access-7462m") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "kube-api-access-7462m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:54.375567 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.375464    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-token" (OuterVolumeSpecName: "istio-token") pod "cbcbefac-4382-4a7a-93ea-9e5492576609" (UID: "cbcbefac-4382-4a7a-93ea-9e5492576609"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:54.474013 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.473977    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbcbefac-4382-4a7a-93ea-9e5492576609-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.474013 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.474006    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.474013 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.474015    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7462m\" (UniqueName: \"kubernetes.io/projected/cbcbefac-4382-4a7a-93ea-9e5492576609-kube-api-access-7462m\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.474013 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.474024    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.474282 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.474033    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbcbefac-4382-4a7a-93ea-9e5492576609-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:54.746168 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.746123    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"
Sep 09 13:07:54.746345 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.746169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5" event={"ID":"1f6ba2e9-99ea-4a46-adfb-58976b159a99","Type":"ContainerDied","Data":"acdfce60ab5004b44a0d153c00b584f3c53f23d88229a2c4305811fcfd8c80f7"}
Sep 09 13:07:54.746345 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.746229    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acdfce60ab5004b44a0d153c00b584f3c53f23d88229a2c4305811fcfd8c80f7"
Sep 09 13:07:54.747597 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.747578    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"
Sep 09 13:07:54.747749 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.747619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z" event={"ID":"21ee1f6d-d273-4d22-82f6-6b60da8cd438","Type":"ContainerDied","Data":"da6cc25f9bf5482452a025fc4686c3d54f2696f64e3db340d5d5342562f0bb26"}
Sep 09 13:07:54.747749 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.747688    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da6cc25f9bf5482452a025fc4686c3d54f2696f64e3db340d5d5342562f0bb26"
Sep 09 13:07:54.749784 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.749761    2577 generic.go:356] "Generic (PLEG): container finished" podID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerID="4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c" exitCode=0
Sep 09 13:07:54.749882 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.749830    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" event={"ID":"cbcbefac-4382-4a7a-93ea-9e5492576609","Type":"ContainerDied","Data":"4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c"}
Sep 09 13:07:54.749997 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.749900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h" event={"ID":"cbcbefac-4382-4a7a-93ea-9e5492576609","Type":"ContainerDied","Data":"4ce787330e29138326a4ad283350bf603fa8d70c744e531a59e2dbc645d16c59"}
Sep 09 13:07:54.749997 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.749930    2577 scope.go:117] "RemoveContainer" containerID="4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c"
Sep 09 13:07:54.750117 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.749841    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"
Sep 09 13:07:54.759161 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.759144    2577 scope.go:117] "RemoveContainer" containerID="b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa"
Sep 09 13:07:54.768950 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.768932    2577 scope.go:117] "RemoveContainer" containerID="4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c"
Sep 09 13:07:54.769251 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:54.769233    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c\": container with ID starting with 4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c not found: ID does not exist" containerID="4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c"
Sep 09 13:07:54.769326 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.769259    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c"} err="failed to get container status \"4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c\": rpc error: code = NotFound desc = could not find container \"4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c\": container with ID starting with 4a477c2a32520f0ba98dc1b0a7bbafa6446f4f70247d1a52cf75313b30a70a2c not found: ID does not exist"
Sep 09 13:07:54.769326 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.769277    2577 scope.go:117] "RemoveContainer" containerID="b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa"
Sep 09 13:07:54.769538 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:54.769515    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa\": container with ID starting with b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa not found: ID does not exist" containerID="b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa"
Sep 09 13:07:54.769602 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.769546    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa"} err="failed to get container status \"b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa\": rpc error: code = NotFound desc = could not find container \"b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa\": container with ID starting with b58057735c8d2e4fbfd69a6f0cf49c90a70df06705640094228e0a0fd5863cfa not found: ID does not exist"
Sep 09 13:07:54.774162 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.774134    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"]
Sep 09 13:07:54.779310 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:54.779284    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3b4b96-openshift-default-6875bcc4-5rw6h"]
Sep 09 13:07:56.515805 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:56.515704    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" path="/var/lib/kubelet/pods/cbcbefac-4382-4a7a-93ea-9e5492576609/volumes"
Sep 09 13:07:58.467502 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.467459    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"]
Sep 09 13:07:58.468002 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.467743    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97" podUID="4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563" containerName="payload-processing" containerID="cri-o://6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22" gracePeriod=30
Sep 09 13:07:58.478156 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.478119    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"]
Sep 09 13:07:58.478406 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.478373    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp" podUID="8caa9ac8-71a8-4f31-bdf7-cdeebe755576" containerName="payload-pre-processing" containerID="cri-o://d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40" gracePeriod=30
Sep 09 13:07:58.634888 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.634609    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"]
Sep 09 13:07:58.637310 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.637281    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-73011135-gzt4z"]
Sep 09 13:07:58.758195 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.758168    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:58.760241 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.760223    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:58.775242 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.775196    2577 generic.go:356] "Generic (PLEG): container finished" podID="4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563" containerID="6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22" exitCode=0
Sep 09 13:07:58.775463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.775316    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"
Sep 09 13:07:58.775463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.775345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97" event={"ID":"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563","Type":"ContainerDied","Data":"6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22"}
Sep 09 13:07:58.775463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.775399    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97" event={"ID":"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563","Type":"ContainerDied","Data":"7e10d1f477e99387afb7360dfd3378ee5e9e4be91c13c5db22b651b9078e89e5"}
Sep 09 13:07:58.775463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.775430    2577 scope.go:117] "RemoveContainer" containerID="6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22"
Sep 09 13:07:58.777353 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.777328    2577 generic.go:356] "Generic (PLEG): container finished" podID="8caa9ac8-71a8-4f31-bdf7-cdeebe755576" containerID="d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40" exitCode=0
Sep 09 13:07:58.777479 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.777367    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"
Sep 09 13:07:58.777479 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.777394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp" event={"ID":"8caa9ac8-71a8-4f31-bdf7-cdeebe755576","Type":"ContainerDied","Data":"d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40"}
Sep 09 13:07:58.777479 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.777437    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp" event={"ID":"8caa9ac8-71a8-4f31-bdf7-cdeebe755576","Type":"ContainerDied","Data":"bfcc9110fbe132d4acd2ea7ae5a1c985c8497bf42f37f30a155f74a5d86e3ba9"}
Sep 09 13:07:58.785284 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.785261    2577 scope.go:117] "RemoveContainer" containerID="6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22"
Sep 09 13:07:58.785723 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:58.785701    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22\": container with ID starting with 6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22 not found: ID does not exist" containerID="6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22"
Sep 09 13:07:58.785841 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.785747    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22"} err="failed to get container status \"6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22\": rpc error: code = NotFound desc = could not find container \"6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22\": container with ID starting with 6b202a4f7fbc0905f031011a753297069ca24ea4238ac0318a0a928130658c22 not found: ID does not exist"
Sep 09 13:07:58.785841 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.785770    2577 scope.go:117] "RemoveContainer" containerID="d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40"
Sep 09 13:07:58.797842 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.797812    2577 scope.go:117] "RemoveContainer" containerID="d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40"
Sep 09 13:07:58.798313 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:07:58.798275    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40\": container with ID starting with d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40 not found: ID does not exist" containerID="d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40"
Sep 09 13:07:58.798433 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.798325    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40"} err="failed to get container status \"d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40\": rpc error: code = NotFound desc = could not find container \"d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40\": container with ID starting with d7e39155f7bec105a4638ef3ad1c4e160c1945ddd12dc358f7a5fb6377868c40 not found: ID does not exist"
Sep 09 13:07:58.812831 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.812791    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-plugins-config-volume\") pod \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\" (UID: \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\") "
Sep 09 13:07:58.813479 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.812932    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c89rd\" (UniqueName: \"kubernetes.io/projected/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-kube-api-access-c89rd\") pod \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\" (UID: \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\") "
Sep 09 13:07:58.813479 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.812989    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbht2\" (UniqueName: \"kubernetes.io/projected/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-kube-api-access-mbht2\") pod \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\" (UID: \"4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563\") "
Sep 09 13:07:58.813479 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.813017    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-plugins-config-volume\") pod \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\" (UID: \"8caa9ac8-71a8-4f31-bdf7-cdeebe755576\") "
Sep 09 13:07:58.813479 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.813250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563" (UID: "4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:58.813789 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.813589    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8caa9ac8-71a8-4f31-bdf7-cdeebe755576" (UID: "8caa9ac8-71a8-4f31-bdf7-cdeebe755576"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:07:58.815781 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.815751    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-kube-api-access-c89rd" (OuterVolumeSpecName: "kube-api-access-c89rd") pod "8caa9ac8-71a8-4f31-bdf7-cdeebe755576" (UID: "8caa9ac8-71a8-4f31-bdf7-cdeebe755576"). InnerVolumeSpecName "kube-api-access-c89rd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:58.815906 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.815844    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-kube-api-access-mbht2" (OuterVolumeSpecName: "kube-api-access-mbht2") pod "4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563" (UID: "4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563"). InnerVolumeSpecName "kube-api-access-mbht2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:58.914539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.914504    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbht2\" (UniqueName: \"kubernetes.io/projected/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-kube-api-access-mbht2\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:58.914539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.914530    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:58.914539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.914541    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:58.914539 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:58.914550    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c89rd\" (UniqueName: \"kubernetes.io/projected/8caa9ac8-71a8-4f31-bdf7-cdeebe755576-kube-api-access-c89rd\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:07:59.098897 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:59.098873    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"]
Sep 09 13:07:59.102572 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:59.102550    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-73011135-5956769d6b-nmssp"]
Sep 09 13:07:59.121838 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:59.121813    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"]
Sep 09 13:07:59.125167 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:07:59.125147    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-73011135-588566f8cc-rhd97"]
Sep 09 13:08:00.202517 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.202483    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"]
Sep 09 13:08:00.202979 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.202811    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk" podUID="dd066d77-17a4-4bdb-b495-316ae86d9a1b" containerName="payload-pre-processing" containerID="cri-o://74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8" gracePeriod=30
Sep 09 13:08:00.362177 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.362138    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"]
Sep 09 13:08:00.365685 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.365651    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7c991b-rjhp5"]
Sep 09 13:08:00.463995 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.463934    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:08:00.515491 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.515458    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f6ba2e9-99ea-4a46-adfb-58976b159a99" path="/var/lib/kubelet/pods/1f6ba2e9-99ea-4a46-adfb-58976b159a99/volumes"
Sep 09 13:08:00.515846 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.515831    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21ee1f6d-d273-4d22-82f6-6b60da8cd438" path="/var/lib/kubelet/pods/21ee1f6d-d273-4d22-82f6-6b60da8cd438/volumes"
Sep 09 13:08:00.516147 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.516134    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563" path="/var/lib/kubelet/pods/4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563/volumes"
Sep 09 13:08:00.516437 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.516426    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8caa9ac8-71a8-4f31-bdf7-cdeebe755576" path="/var/lib/kubelet/pods/8caa9ac8-71a8-4f31-bdf7-cdeebe755576/volumes"
Sep 09 13:08:00.528595 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.528573    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd066d77-17a4-4bdb-b495-316ae86d9a1b-plugins-config-volume\") pod \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\" (UID: \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\") "
Sep 09 13:08:00.528696 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.528687    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqzdn\" (UniqueName: \"kubernetes.io/projected/dd066d77-17a4-4bdb-b495-316ae86d9a1b-kube-api-access-bqzdn\") pod \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\" (UID: \"dd066d77-17a4-4bdb-b495-316ae86d9a1b\") "
Sep 09 13:08:00.528901 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.528873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd066d77-17a4-4bdb-b495-316ae86d9a1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd066d77-17a4-4bdb-b495-316ae86d9a1b" (UID: "dd066d77-17a4-4bdb-b495-316ae86d9a1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:08:00.530901 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.530878    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd066d77-17a4-4bdb-b495-316ae86d9a1b-kube-api-access-bqzdn" (OuterVolumeSpecName: "kube-api-access-bqzdn") pod "dd066d77-17a4-4bdb-b495-316ae86d9a1b" (UID: "dd066d77-17a4-4bdb-b495-316ae86d9a1b"). InnerVolumeSpecName "kube-api-access-bqzdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:08:00.629420 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.629395    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqzdn\" (UniqueName: \"kubernetes.io/projected/dd066d77-17a4-4bdb-b495-316ae86d9a1b-kube-api-access-bqzdn\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:00.629420 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.629417    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd066d77-17a4-4bdb-b495-316ae86d9a1b-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:00.790258 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.790184    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd066d77-17a4-4bdb-b495-316ae86d9a1b" containerID="74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8" exitCode=0
Sep 09 13:08:00.790258 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.790233    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"
Sep 09 13:08:00.790446 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.790263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk" event={"ID":"dd066d77-17a4-4bdb-b495-316ae86d9a1b","Type":"ContainerDied","Data":"74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8"}
Sep 09 13:08:00.790446 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.790315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk" event={"ID":"dd066d77-17a4-4bdb-b495-316ae86d9a1b","Type":"ContainerDied","Data":"d48add5b04ad841129421c3cc6cc094313b6cc15f9b6b150676b8aa62da43617"}
Sep 09 13:08:00.790446 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.790336    2577 scope.go:117] "RemoveContainer" containerID="74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8"
Sep 09 13:08:00.799028 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.799015    2577 scope.go:117] "RemoveContainer" containerID="74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8"
Sep 09 13:08:00.799294 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:00.799276    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8\": container with ID starting with 74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8 not found: ID does not exist" containerID="74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8"
Sep 09 13:08:00.799357 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.799301    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8"} err="failed to get container status \"74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8\": rpc error: code = NotFound desc = could not find container \"74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8\": container with ID starting with 74a16dc36131882f2170c5bcd32314853c7a9edf2f4a2ffce262c095e55de1e8 not found: ID does not exist"
Sep 09 13:08:00.810805 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.810647    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"]
Sep 09 13:08:00.812347 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:00.812329    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7c991b-5d7ffd65d6-7tplk"]
Sep 09 13:08:02.514941 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:02.514907    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd066d77-17a4-4bdb-b495-316ae86d9a1b" path="/var/lib/kubelet/pods/dd066d77-17a4-4bdb-b495-316ae86d9a1b/volumes"
Sep 09 13:08:04.925040 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:04.924999    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"]
Sep 09 13:08:04.925540 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:04.925262    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy" containerID="cri-o://d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5" gracePeriod=30
Sep 09 13:08:08.291696 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.291664    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"]
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292006    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292022    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292041    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563" containerName="payload-processing"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292046    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563" containerName="payload-processing"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292055    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21ee1f6d-d273-4d22-82f6-6b60da8cd438" containerName="revoke-keys"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292061    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="21ee1f6d-d273-4d22-82f6-6b60da8cd438" containerName="revoke-keys"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292067    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd066d77-17a4-4bdb-b495-316ae86d9a1b" containerName="payload-pre-processing"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292071    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd066d77-17a4-4bdb-b495-316ae86d9a1b" containerName="payload-pre-processing"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292081    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8caa9ac8-71a8-4f31-bdf7-cdeebe755576" containerName="payload-pre-processing"
Sep 09 13:08:08.292095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292086    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8caa9ac8-71a8-4f31-bdf7-cdeebe755576" containerName="payload-pre-processing"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292105    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f6ba2e9-99ea-4a46-adfb-58976b159a99" containerName="revoke-keys"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292110    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f6ba2e9-99ea-4a46-adfb-58976b159a99" containerName="revoke-keys"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292158    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f6ba2e9-99ea-4a46-adfb-58976b159a99" containerName="revoke-keys"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292169    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd066d77-17a4-4bdb-b495-316ae86d9a1b" containerName="payload-pre-processing"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292179    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d65ac31-43c6-4ddf-8ed7-8c38a4e5a563" containerName="payload-processing"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292187    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8caa9ac8-71a8-4f31-bdf7-cdeebe755576" containerName="payload-pre-processing"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292196    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="21ee1f6d-d273-4d22-82f6-6b60da8cd438" containerName="revoke-keys"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292204    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy"
Sep 09 13:08:08.292463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.292209    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy"
Sep 09 13:08:08.295153 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.295135    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.297588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.297552    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-ada1ae-openshift-default-dockercfg-mdpmn\""
Sep 09 13:08:08.305843 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.305820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"]
Sep 09 13:08:08.399565 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-envoy\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.399795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399583    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-credential-socket\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.399795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-istio-token\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.399795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-certs\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.399795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-socket\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.400025 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f8514ef-8d13-479e-a124-c494753c35b8-istio-podinfo\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.400025 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg6vg\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-kube-api-access-xg6vg\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.400025 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f8514ef-8d13-479e-a124-c494753c35b8-istiod-ca-cert\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.400025 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.399936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-data\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501091 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-data\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501291 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-envoy\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501291 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-credential-socket\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501291 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-istio-token\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501291 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-certs\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501291 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-socket\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501291 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f8514ef-8d13-479e-a124-c494753c35b8-istio-podinfo\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501603 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501305    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg6vg\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-kube-api-access-xg6vg\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501603 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f8514ef-8d13-479e-a124-c494753c35b8-istiod-ca-cert\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501603 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-data\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501811 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-credential-socket\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501811 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501727    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-socket\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.501811 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.501798    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-certs\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.502066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.502048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f8514ef-8d13-479e-a124-c494753c35b8-istiod-ca-cert\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.503552 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.503534    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-envoy\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.503993 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.503975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f8514ef-8d13-479e-a124-c494753c35b8-istio-podinfo\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.508904 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.508879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-istio-token\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.509033 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.508921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg6vg\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-kube-api-access-xg6vg\") pod \"e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") " pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.606398 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.606307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:08.883012 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.882977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"]
Sep 09 13:08:08.885919 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:08.885890    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f8514ef_8d13_479e_a124_c494753c35b8.slice/crio-7c4247c26c3831d8facf582709d929934c62f085f75d341f18858a5e821d0914 WatchSource:0}: Error finding container 7c4247c26c3831d8facf582709d929934c62f085f75d341f18858a5e821d0914: Status 404 returned error can't find the container with id 7c4247c26c3831d8facf582709d929934c62f085f75d341f18858a5e821d0914
Sep 09 13:08:08.888149 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.888115    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:08.888260 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.888202    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:08.888260 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:08.888249    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:09.834247 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:09.834212    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" event={"ID":"5f8514ef-8d13-479e-a124-c494753c35b8","Type":"ContainerStarted","Data":"0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9"}
Sep 09 13:08:09.834658 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:09.834267    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" event={"ID":"5f8514ef-8d13-479e-a124-c494753c35b8","Type":"ContainerStarted","Data":"7c4247c26c3831d8facf582709d929934c62f085f75d341f18858a5e821d0914"}
Sep 09 13:08:09.855964 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:09.855910    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podStartSLOduration=1.855893752 podStartE2EDuration="1.855893752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:09.854662645 +0000 UTC m=+1175.855328742" watchObservedRunningTime="2026-09-09 13:08:09.855893752 +0000 UTC m=+1175.856559842"
Sep 09 13:08:10.183559 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.183533    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:08:10.320266 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320229    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fad17ac-5b46-47ac-bed4-80ea534aa377-istiod-ca-cert\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.320428 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320287    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-socket\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.320428 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320313    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-envoy\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.320542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320479    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-credential-socket\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.320542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320525    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c62mv\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-kube-api-access-c62mv\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.320694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320542    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-data\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.320694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320595    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-token\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.320694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320621    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:08:10.320694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320664    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-certs\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.320897 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320807    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:08:10.320897 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320813    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fad17ac-5b46-47ac-bed4-80ea534aa377-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:08:10.320897 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320860    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-data" (OuterVolumeSpecName: "istio-data") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:08:10.321061 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320894    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:08:10.321061 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.320723    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-podinfo\") pod \"0fad17ac-5b46-47ac-bed4-80ea534aa377\" (UID: \"0fad17ac-5b46-47ac-bed4-80ea534aa377\") "
Sep 09 13:08:10.321374 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.321354    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fad17ac-5b46-47ac-bed4-80ea534aa377-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.321425 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.321380    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.321425 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.321394    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.321425 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.321408    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.321425 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.321421    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.322918 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.322894    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-token" (OuterVolumeSpecName: "istio-token") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:08:10.322918 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.322893    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:08:10.323316 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.323292    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-kube-api-access-c62mv" (OuterVolumeSpecName: "kube-api-access-c62mv") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "kube-api-access-c62mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:08:10.323316 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.323298    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0fad17ac-5b46-47ac-bed4-80ea534aa377" (UID: "0fad17ac-5b46-47ac-bed4-80ea534aa377"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:08:10.423299 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.422979    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c62mv\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-kube-api-access-c62mv\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.423299 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.423050    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.423299 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.423070    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.423299 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.423091    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fad17ac-5b46-47ac-bed4-80ea534aa377-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:08:10.607063 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.607020    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:10.612417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.612390    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:10.840831 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.840746    2577 generic.go:356] "Generic (PLEG): container finished" podID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerID="d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5" exitCode=0
Sep 09 13:08:10.840831 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.840824    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"
Sep 09 13:08:10.841290 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.840839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" event={"ID":"0fad17ac-5b46-47ac-bed4-80ea534aa377","Type":"ContainerDied","Data":"d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5"}
Sep 09 13:08:10.841290 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.840900    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5" event={"ID":"0fad17ac-5b46-47ac-bed4-80ea534aa377","Type":"ContainerDied","Data":"a488a314b6bcf48675cb98c0e16fc72774d8c5b3ee277467fe146a5bbca451d0"}
Sep 09 13:08:10.841290 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.841240    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:10.841799 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.840929    2577 scope.go:117] "RemoveContainer" containerID="d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5"
Sep 09 13:08:10.842316 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.842293    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:10.850907 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.850891    2577 scope.go:117] "RemoveContainer" containerID="680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae"
Sep 09 13:08:10.856063 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.856042    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"]
Sep 09 13:08:10.860524 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.860482    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-73011135-openshift-default-6f6ffd7c8d-q99j5"]
Sep 09 13:08:10.860597 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.860542    2577 scope.go:117] "RemoveContainer" containerID="d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5"
Sep 09 13:08:10.860883 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:10.860860    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5\": container with ID starting with d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5 not found: ID does not exist" containerID="d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5"
Sep 09 13:08:10.860956 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.860890    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5"} err="failed to get container status \"d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5\": rpc error: code = NotFound desc = could not find container \"d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5\": container with ID starting with d3a4f4d48dd0953f6982b8efcf654d59120c3aeb6b775bf55c84251a6b96fce5 not found: ID does not exist"
Sep 09 13:08:10.860956 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.860910    2577 scope.go:117] "RemoveContainer" containerID="680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae"
Sep 09 13:08:10.861180 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:10.861163    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae\": container with ID starting with 680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae not found: ID does not exist" containerID="680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae"
Sep 09 13:08:10.861222 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:10.861186    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae"} err="failed to get container status \"680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae\": rpc error: code = NotFound desc = could not find container \"680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae\": container with ID starting with 680d5f28524e951997ffb05340451f5d6aeaadd20285868b6ae28af4982084ae not found: ID does not exist"
Sep 09 13:08:12.515331 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.515294    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" path="/var/lib/kubelet/pods/0fad17ac-5b46-47ac-bed4-80ea534aa377/volumes"
Sep 09 13:08:12.992432 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.992392    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"]
Sep 09 13:08:12.992890 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.992866    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy"
Sep 09 13:08:12.992890 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.992890    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy"
Sep 09 13:08:12.993072 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.992926    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy"
Sep 09 13:08:12.993072 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.992935    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy"
Sep 09 13:08:12.993072 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.992946    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy"
Sep 09 13:08:12.993072 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.992954    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbcbefac-4382-4a7a-93ea-9e5492576609" containerName="istio-proxy"
Sep 09 13:08:12.993072 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.993024    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy"
Sep 09 13:08:12.997778 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:12.997758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.000341 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.000319    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 13:08:13.001388 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.001254    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dbjk4\""
Sep 09 13:08:13.001388 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.001254    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-ada1ae-serving-cert\""
Sep 09 13:08:13.001388 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.001350    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 13:08:13.001388 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.001360    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 13:08:13.006492 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.006452    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"]
Sep 09 13:08:13.150526 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.150491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84hls\" (UniqueName: \"kubernetes.io/projected/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-kube-api-access-84hls\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.150720 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.150543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.150720 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.150626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.150720 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.150689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.251112 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.251040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84hls\" (UniqueName: \"kubernetes.io/projected/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-kube-api-access-84hls\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.251112 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.251079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.251282 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.251199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.251282 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.251239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.251365 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:13.251345    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-serving-cert: secret "maas-api-e2e-shared-a-w3-ada1ae-serving-cert" not found
Sep 09 13:08:13.251415 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:13.251405    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls podName:ae81c7bd-b21f-4a19-aa8d-92fbf0563f67 nodeName:}" failed. No retries permitted until 2026-09-09 13:08:13.751386663 +0000 UTC m=+1179.752052741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls") pod "maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg" (UID: "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67") : secret "maas-api-e2e-shared-a-w3-ada1ae-serving-cert" not found
Sep 09 13:08:13.251728 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.251701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.253826 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.253801    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.259475 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.259451    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84hls\" (UniqueName: \"kubernetes.io/projected/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-kube-api-access-84hls\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.609986 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.609903    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"]
Sep 09 13:08:13.610360 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.610278    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fad17ac-5b46-47ac-bed4-80ea534aa377" containerName="istio-proxy"
Sep 09 13:08:13.613362 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.613346    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:13.615513 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.615490    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-ada1ae-dockercfg-7jzdk\""
Sep 09 13:08:13.615653 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.615499    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-ada1ae\""
Sep 09 13:08:13.622917 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.622894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"]
Sep 09 13:08:13.756887 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.756857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.757073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.756898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bstzh\" (UniqueName: \"kubernetes.io/projected/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-kube-api-access-bstzh\") pod \"payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg\" (UID: \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:13.757073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.756936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg\" (UID: \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:13.759409 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.759390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.858085 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.858061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bstzh\" (UniqueName: \"kubernetes.io/projected/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-kube-api-access-bstzh\") pod \"payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg\" (UID: \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:13.858236 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.858095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg\" (UID: \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:13.858751 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.858733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg\" (UID: \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:13.865889 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.865839    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bstzh\" (UniqueName: \"kubernetes.io/projected/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-kube-api-access-bstzh\") pod \"payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg\" (UID: \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:13.909847 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.909827    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:13.924689 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:13.924668    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:14.180505 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.180480    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"]
Sep 09 13:08:14.182394 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:14.182371    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae81c7bd_b21f_4a19_aa8d_92fbf0563f67.slice/crio-b0f71bf79435d81bdd2de1a061279e0bc7c619085bdc01b8570e3e6298307b91 WatchSource:0}: Error finding container b0f71bf79435d81bdd2de1a061279e0bc7c619085bdc01b8570e3e6298307b91: Status 404 returned error can't find the container with id b0f71bf79435d81bdd2de1a061279e0bc7c619085bdc01b8570e3e6298307b91
Sep 09 13:08:14.213098 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.213077    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"]
Sep 09 13:08:14.214053 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:14.214025    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1a66c9_4cfa_4c47_9a41_51b7c5a9a259.slice/crio-473bb1a528021471908443a9bdfb7d0e10012375b75629d16d91e2bb9affb486 WatchSource:0}: Error finding container 473bb1a528021471908443a9bdfb7d0e10012375b75629d16d91e2bb9affb486: Status 404 returned error can't find the container with id 473bb1a528021471908443a9bdfb7d0e10012375b75629d16d91e2bb9affb486
Sep 09 13:08:14.862175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.862077    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg" event={"ID":"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67","Type":"ContainerStarted","Data":"10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024"}
Sep 09 13:08:14.862175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.862145    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:14.862175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.862167    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg" event={"ID":"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67","Type":"ContainerStarted","Data":"b0f71bf79435d81bdd2de1a061279e0bc7c619085bdc01b8570e3e6298307b91"}
Sep 09 13:08:14.863687 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.863661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg" event={"ID":"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259","Type":"ContainerStarted","Data":"bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034"}
Sep 09 13:08:14.863836 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.863707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg" event={"ID":"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259","Type":"ContainerStarted","Data":"473bb1a528021471908443a9bdfb7d0e10012375b75629d16d91e2bb9affb486"}
Sep 09 13:08:14.863836 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.863733    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:14.889769 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.889718    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg" podStartSLOduration=2.5656161490000002 podStartE2EDuration="2.889700099s" totalImagesPullingTime="324.08395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:14.887372703 +0000 UTC m=+1180.888038800" watchObservedRunningTime="2026-09-09 13:08:14.889700099 +0000 UTC m=+1180.890366204"
Sep 09 13:08:14.917131 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:14.917094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg" podStartSLOduration=1.917080818 podStartE2EDuration="1.917080818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:14.914326328 +0000 UTC m=+1180.914992430" watchObservedRunningTime="2026-09-09 13:08:14.917080818 +0000 UTC m=+1180.917746919"
Sep 09 13:08:18.909769 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:18.909735    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"]
Sep 09 13:08:18.913356 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:18.913336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:18.915839 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:18.915815    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-0b0a1a-openshift-default-dockercfg-hg5qw\""
Sep 09 13:08:18.926774 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:18.926751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"]
Sep 09 13:08:19.006511 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-credential-socket\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.006730 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-token\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.006730 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-podinfo\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.006730 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-socket\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.006730 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zcfr\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-kube-api-access-6zcfr\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.006956 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-certs\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.006956 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b91e4-6ada-44fb-a5be-e10475c24cbe-istiod-ca-cert\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.006956 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-envoy\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.006956 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.006867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-data\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-credential-socket\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107474    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-token\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107723 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-podinfo\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107723 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-socket\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107723 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zcfr\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-kube-api-access-6zcfr\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107859 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-certs\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107859 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b91e4-6ada-44fb-a5be-e10475c24cbe-istiod-ca-cert\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107859 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-envoy\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.107859 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-data\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.108043 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-socket\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.108043 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.107984    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-credential-socket\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.108252 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.108232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-data\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.108438 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.108304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-certs\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.108913 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.108883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b91e4-6ada-44fb-a5be-e10475c24cbe-istiod-ca-cert\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.110241 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.110218    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-envoy\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.110324 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.110311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-podinfo\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.117920 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.117899    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-token\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.117920 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.117912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zcfr\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-kube-api-access-6zcfr\") pod \"e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") " pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.223730 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.223625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:19.520499 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.520478    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"]
Sep 09 13:08:19.522160 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:19.522130    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988b91e4_6ada_44fb_a5be_e10475c24cbe.slice/crio-c4bb2e312ce9459ea4daa5d186a91abb9f17761c974ee136c06d8dc64cf168a7 WatchSource:0}: Error finding container c4bb2e312ce9459ea4daa5d186a91abb9f17761c974ee136c06d8dc64cf168a7: Status 404 returned error can't find the container with id c4bb2e312ce9459ea4daa5d186a91abb9f17761c974ee136c06d8dc64cf168a7
Sep 09 13:08:19.524426 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.524398    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:19.524481 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.524465    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:19.524518 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.524491    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:19.898006 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.897966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" event={"ID":"988b91e4-6ada-44fb-a5be-e10475c24cbe","Type":"ContainerStarted","Data":"be7e63b1be70bc63b1b9e29347675e636729e524de6eb242c662a89d27bf59fc"}
Sep 09 13:08:19.898209 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.898019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" event={"ID":"988b91e4-6ada-44fb-a5be-e10475c24cbe","Type":"ContainerStarted","Data":"c4bb2e312ce9459ea4daa5d186a91abb9f17761c974ee136c06d8dc64cf168a7"}
Sep 09 13:08:19.919009 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:19.918956    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" podStartSLOduration=1.918937342 podStartE2EDuration="1.918937342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:19.916110487 +0000 UTC m=+1185.916776764" watchObservedRunningTime="2026-09-09 13:08:19.918937342 +0000 UTC m=+1185.919603446"
Sep 09 13:08:20.224894 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:20.224804    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:20.229390 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:20.229368    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:20.885918 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:20.885884    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:08:20.903364 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:20.903322    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:20.904271 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:20.904251    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:23.077197 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.077156    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"]
Sep 09 13:08:23.080786 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.080765    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.083190 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.083166    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-0b0a1a-serving-cert\""
Sep 09 13:08:23.090313 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.090285    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"]
Sep 09 13:08:23.242558 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.242524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.242761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.242575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.242761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.242660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a42376a-140f-4b99-a2d2-e218ead49187-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.242761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.242721    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs4fq\" (UniqueName: \"kubernetes.io/projected/9a42376a-140f-4b99-a2d2-e218ead49187-kube-api-access-gs4fq\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.343795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.343716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.343795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.343772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.344042 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.343811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a42376a-140f-4b99-a2d2-e218ead49187-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.344042 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.343885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4fq\" (UniqueName: \"kubernetes.io/projected/9a42376a-140f-4b99-a2d2-e218ead49187-kube-api-access-gs4fq\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.344042 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:23.343939    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-serving-cert: secret "maas-api-e2e-shared-a-w2-0b0a1a-serving-cert" not found
Sep 09 13:08:23.344042 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:23.344035    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls podName:9a42376a-140f-4b99-a2d2-e218ead49187 nodeName:}" failed. No retries permitted until 2026-09-09 13:08:23.844011997 +0000 UTC m=+1189.844678078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls") pod "maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b" (UID: "9a42376a-140f-4b99-a2d2-e218ead49187") : secret "maas-api-e2e-shared-a-w2-0b0a1a-serving-cert" not found
Sep 09 13:08:23.344612 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.344587    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a42376a-140f-4b99-a2d2-e218ead49187-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.346569 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.346544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.352481 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.352457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs4fq\" (UniqueName: \"kubernetes.io/projected/9a42376a-140f-4b99-a2d2-e218ead49187-kube-api-access-gs4fq\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.848889 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.848852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.851389 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.851363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:23.996340 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:23.996295    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:24.269025 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:24.268980    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"]
Sep 09 13:08:24.271157 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:24.271130    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a42376a_140f_4b99_a2d2_e218ead49187.slice/crio-e71a2479ea2e4defa2d49c94bb448ba2365a5ae37b8f895f826887c1c760da35 WatchSource:0}: Error finding container e71a2479ea2e4defa2d49c94bb448ba2365a5ae37b8f895f826887c1c760da35: Status 404 returned error can't find the container with id e71a2479ea2e4defa2d49c94bb448ba2365a5ae37b8f895f826887c1c760da35
Sep 09 13:08:24.926379 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:24.925963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b" event={"ID":"9a42376a-140f-4b99-a2d2-e218ead49187","Type":"ContainerStarted","Data":"e71a2479ea2e4defa2d49c94bb448ba2365a5ae37b8f895f826887c1c760da35"}
Sep 09 13:08:25.842836 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.842744    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 13:08:25.843217 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.842815    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 13:08:25.870820 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.870791    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:08:25.933818 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.933787    2577 generic.go:356] "Generic (PLEG): container finished" podID="5f8514ef-8d13-479e-a124-c494753c35b8" containerID="0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9" exitCode=0
Sep 09 13:08:25.934015 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.933864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" event={"ID":"5f8514ef-8d13-479e-a124-c494753c35b8","Type":"ContainerDied","Data":"0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9"}
Sep 09 13:08:25.934384 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.934356    2577 scope.go:117] "RemoveContainer" containerID="0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9"
Sep 09 13:08:25.935628 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.935595    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:25.935763 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.935715    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:25.935829 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.935769    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:25.935897 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.935822    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b" event={"ID":"9a42376a-140f-4b99-a2d2-e218ead49187","Type":"ContainerStarted","Data":"81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565"}
Sep 09 13:08:25.935981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.935966    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:25.967056 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:25.967005    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b" podStartSLOduration=2.3512600470000002 podStartE2EDuration="2.966988998s" totalImagesPullingTime="615.728951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:25.964614344 +0000 UTC m=+1191.965280446" watchObservedRunningTime="2026-09-09 13:08:25.966988998 +0000 UTC m=+1191.967655098"
Sep 09 13:08:26.943555 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:26.943522    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" event={"ID":"5f8514ef-8d13-479e-a124-c494753c35b8","Type":"ContainerStarted","Data":"f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab"}
Sep 09 13:08:27.606896 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:27.606849    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:27.607151 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:27.607128    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 13:08:27.607198 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:27.607181    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 13:08:28.607429 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:28.607377    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 13:08:28.608010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:28.607453    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 13:08:29.607926 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:29.607887    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 13:08:29.608561 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:29.608527    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 13:08:30.607566 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:30.607523    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 13:08:30.607909 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:30.607874    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 13:08:31.607468 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:31.607416    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 13:08:31.608120 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:31.608070    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 13:08:31.949785 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:31.949754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:08:32.611386 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:32.611351    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:32.611799 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:32.611627    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:32.612352 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:32.612337    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:08:36.630305 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:36.630270    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:55640->10.133.0.53:15021: read: connection reset by peer" start-of-body=
Sep 09 13:08:36.630844 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:36.630346    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:55640->10.133.0.53:15021: read: connection reset by peer"
Sep 09 13:08:37.001649 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.001540    2577 generic.go:356] "Generic (PLEG): container finished" podID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerID="be7e63b1be70bc63b1b9e29347675e636729e524de6eb242c662a89d27bf59fc" exitCode=0
Sep 09 13:08:37.001649 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.001622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" event={"ID":"988b91e4-6ada-44fb-a5be-e10475c24cbe","Type":"ContainerDied","Data":"be7e63b1be70bc63b1b9e29347675e636729e524de6eb242c662a89d27bf59fc"}
Sep 09 13:08:37.002060 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.002045    2577 scope.go:117] "RemoveContainer" containerID="be7e63b1be70bc63b1b9e29347675e636729e524de6eb242c662a89d27bf59fc"
Sep 09 13:08:37.003127 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.003097    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:37.003195 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.003187    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:37.003231 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.003220    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:37.523617 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.523578    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"]
Sep 09 13:08:37.526653 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.526610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.529187 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.529166    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-650a99-openshift-default-dockercfg-7ckzr\""
Sep 09 13:08:37.539443 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.539420    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"]
Sep 09 13:08:37.592169 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-data\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.592169 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-token\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.592477 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-podinfo\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.592477 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2ckn\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-kube-api-access-t2ckn\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.592477 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-socket\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.592477 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-envoy\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.592477 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-certs\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.592477 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istiod-ca-cert\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.592477 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.592460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-credential-socket\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.693599 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.693562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istiod-ca-cert\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694106 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.693604    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-credential-socket\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694106 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.693700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-data\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694106 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.693732    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-token\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694106 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.693861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-podinfo\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694106 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.693911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2ckn\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-kube-api-access-t2ckn\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694106 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.693980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-socket\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694106 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.694015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-envoy\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694106 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.694058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-certs\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694538 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.694161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-credential-socket\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694538 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.694233    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-data\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694538 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.694408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-socket\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694538 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.694454    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istiod-ca-cert\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.694769 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.694553    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-certs\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.696702 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.696678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-envoy\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.697139 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.697119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-podinfo\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.701566 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.701545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-token\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.701705 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.701574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2ckn\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-kube-api-access-t2ckn\") pod \"e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") " pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:37.839726 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:37.839621    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:38.008510 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:38.008473    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" event={"ID":"988b91e4-6ada-44fb-a5be-e10475c24cbe","Type":"ContainerStarted","Data":"0962b6041747493f0d929de38d231fa18b3110a1de57bee5d027de1d30517433"}
Sep 09 13:08:38.134747 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:38.134711    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"]
Sep 09 13:08:38.139566 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:38.139539    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8293f6_36cd_4fa4_bbf8_23a7f30d092e.slice/crio-a8fdcb987cf43b5ff5f20b9524fde9109392a6dd2904bb4784e54ecaa8e4b1fc WatchSource:0}: Error finding container a8fdcb987cf43b5ff5f20b9524fde9109392a6dd2904bb4784e54ecaa8e4b1fc: Status 404 returned error can't find the container with id a8fdcb987cf43b5ff5f20b9524fde9109392a6dd2904bb4784e54ecaa8e4b1fc
Sep 09 13:08:38.142478 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:38.142438    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:38.142569 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:38.142533    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:38.142610 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:38.142577    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:38.224871 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:38.224848    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:38.225132 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:38.225105    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 09 13:08:38.225228 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:38.225156    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 13:08:39.014417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:39.014372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" event={"ID":"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e","Type":"ContainerStarted","Data":"3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e"}
Sep 09 13:08:39.014965 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:39.014439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" event={"ID":"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e","Type":"ContainerStarted","Data":"a8fdcb987cf43b5ff5f20b9524fde9109392a6dd2904bb4784e54ecaa8e4b1fc"}
Sep 09 13:08:39.039231 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:39.039177    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podStartSLOduration=2.03916085 podStartE2EDuration="2.03916085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:39.037517149 +0000 UTC m=+1205.038183255" watchObservedRunningTime="2026-09-09 13:08:39.03916085 +0000 UTC m=+1205.039826950"
Sep 09 13:08:39.224885 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:39.224847    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 09 13:08:39.225068 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:39.224904    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 13:08:39.839961 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:39.839927    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:39.844905 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:39.844879    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:40.019474 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:40.019445    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:40.020543 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:40.020523    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:40.224980 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:40.224940    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 09 13:08:40.225163 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:40.224998    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 13:08:41.224805 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:41.224770    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 09 13:08:41.225213 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:41.224854    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 13:08:42.224565 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.224532    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Sep 09 13:08:42.224781 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.224589    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Sep 09 13:08:42.721058 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.721014    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"]
Sep 09 13:08:42.723612 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.723589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:42.726063 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.726040    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-650a99-dockercfg-l7fcs\""
Sep 09 13:08:42.726063 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.726055    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-650a99\""
Sep 09 13:08:42.734437 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.734412    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"]
Sep 09 13:08:42.843332 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.843282    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00174d6-edfc-40eb-ad03-d8f13242eb07-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq\" (UID: \"b00174d6-edfc-40eb-ad03-d8f13242eb07\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:42.843688 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.843658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlsg7\" (UniqueName: \"kubernetes.io/projected/b00174d6-edfc-40eb-ad03-d8f13242eb07-kube-api-access-wlsg7\") pod \"payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq\" (UID: \"b00174d6-edfc-40eb-ad03-d8f13242eb07\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:42.944524 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.944484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00174d6-edfc-40eb-ad03-d8f13242eb07-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq\" (UID: \"b00174d6-edfc-40eb-ad03-d8f13242eb07\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:42.944780 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.944552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlsg7\" (UniqueName: \"kubernetes.io/projected/b00174d6-edfc-40eb-ad03-d8f13242eb07-kube-api-access-wlsg7\") pod \"payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq\" (UID: \"b00174d6-edfc-40eb-ad03-d8f13242eb07\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:42.945254 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.945233    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00174d6-edfc-40eb-ad03-d8f13242eb07-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq\" (UID: \"b00174d6-edfc-40eb-ad03-d8f13242eb07\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:42.952259 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:42.952231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlsg7\" (UniqueName: \"kubernetes.io/projected/b00174d6-edfc-40eb-ad03-d8f13242eb07-kube-api-access-wlsg7\") pod \"payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq\" (UID: \"b00174d6-edfc-40eb-ad03-d8f13242eb07\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:43.035545 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:43.035443    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:43.227555 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:43.227525    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:43.227780 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:43.227760    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:43.228799 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:43.228784    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:08:43.310783 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:43.310733    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"]
Sep 09 13:08:43.313218 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:43.313188    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00174d6_edfc_40eb_ad03_d8f13242eb07.slice/crio-9abcff00247d7ef9e52cc05743a594242e9d346996b54f4a9d23912930512f7d WatchSource:0}: Error finding container 9abcff00247d7ef9e52cc05743a594242e9d346996b54f4a9d23912930512f7d: Status 404 returned error can't find the container with id 9abcff00247d7ef9e52cc05743a594242e9d346996b54f4a9d23912930512f7d
Sep 09 13:08:44.040481 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:44.040450    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq" event={"ID":"b00174d6-edfc-40eb-ad03-d8f13242eb07","Type":"ContainerStarted","Data":"2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6"}
Sep 09 13:08:44.040981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:44.040501    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq" event={"ID":"b00174d6-edfc-40eb-ad03-d8f13242eb07","Type":"ContainerStarted","Data":"9abcff00247d7ef9e52cc05743a594242e9d346996b54f4a9d23912930512f7d"}
Sep 09 13:08:44.040981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:44.040816    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:44.055514 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:44.055429    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq" podStartSLOduration=2.055408681 podStartE2EDuration="2.055408681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:44.055243621 +0000 UTC m=+1210.055909723" watchObservedRunningTime="2026-09-09 13:08:44.055408681 +0000 UTC m=+1210.056074784"
Sep 09 13:08:51.576098 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.576050    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"]
Sep 09 13:08:51.579708 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.579627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.584926 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.584899    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-941953-serving-cert\""
Sep 09 13:08:51.602660 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.602545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"]
Sep 09 13:08:51.631512 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.631474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.631783 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.631761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba8fee37-6c76-4558-b530-4dae1aed42fe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.631968 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.631953    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvwn5\" (UniqueName: \"kubernetes.io/projected/ba8fee37-6c76-4558-b530-4dae1aed42fe-kube-api-access-bvwn5\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.632165 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.632150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.733691 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.733658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.733898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.733715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.733898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.733742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba8fee37-6c76-4558-b530-4dae1aed42fe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.733898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.733822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvwn5\" (UniqueName: \"kubernetes.io/projected/ba8fee37-6c76-4558-b530-4dae1aed42fe-kube-api-access-bvwn5\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.733898 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:51.733886    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-serving-cert: secret "maas-api-e2e-shared-b-w2-941953-serving-cert" not found
Sep 09 13:08:51.734062 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:08:51.733974    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls podName:ba8fee37-6c76-4558-b530-4dae1aed42fe nodeName:}" failed. No retries permitted until 2026-09-09 13:08:52.233953694 +0000 UTC m=+1218.234619775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls") pod "maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws" (UID: "ba8fee37-6c76-4558-b530-4dae1aed42fe") : secret "maas-api-e2e-shared-b-w2-941953-serving-cert" not found
Sep 09 13:08:51.734476 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.734446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba8fee37-6c76-4558-b530-4dae1aed42fe-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.736296 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.736264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:51.743995 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:51.743969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvwn5\" (UniqueName: \"kubernetes.io/projected/ba8fee37-6c76-4558-b530-4dae1aed42fe-kube-api-access-bvwn5\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:52.239070 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.239036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:52.241676 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.241648    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:52.293817 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.293781    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"]
Sep 09 13:08:52.296375 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.296352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:52.299002 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.298978    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-941953-dockercfg-qzb9z\""
Sep 09 13:08:52.299164 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.299020    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-941953\""
Sep 09 13:08:52.308664 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.308629    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"]
Sep 09 13:08:52.340478 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.340446    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jfhb\" (UniqueName: \"kubernetes.io/projected/6807b4d1-145f-436f-8fcf-e88ce3e372e6-kube-api-access-8jfhb\") pod \"payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq\" (UID: \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:52.340668 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.340505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6807b4d1-145f-436f-8fcf-e88ce3e372e6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq\" (UID: \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:52.441987 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.441940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6807b4d1-145f-436f-8fcf-e88ce3e372e6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq\" (UID: \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:52.442199 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.442076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jfhb\" (UniqueName: \"kubernetes.io/projected/6807b4d1-145f-436f-8fcf-e88ce3e372e6-kube-api-access-8jfhb\") pod \"payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq\" (UID: \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:52.442752 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.442733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6807b4d1-145f-436f-8fcf-e88ce3e372e6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq\" (UID: \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:52.450393 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.450361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jfhb\" (UniqueName: \"kubernetes.io/projected/6807b4d1-145f-436f-8fcf-e88ce3e372e6-kube-api-access-8jfhb\") pod \"payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq\" (UID: \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:52.516854 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.516773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:52.609186 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.608981    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:52.802031 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.802004    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"]
Sep 09 13:08:52.803694 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:52.803664    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba8fee37_6c76_4558_b530_4dae1aed42fe.slice/crio-a78cdaccfe38790c47e2b35fbbb48438eccda347d1dc67ecca42b793954a578e WatchSource:0}: Error finding container a78cdaccfe38790c47e2b35fbbb48438eccda347d1dc67ecca42b793954a578e: Status 404 returned error can't find the container with id a78cdaccfe38790c47e2b35fbbb48438eccda347d1dc67ecca42b793954a578e
Sep 09 13:08:52.892850 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:52.892801    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"]
Sep 09 13:08:52.897186 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:08:52.897158    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6807b4d1_145f_436f_8fcf_e88ce3e372e6.slice/crio-11bd64d90c2f3c39528ff29557399c58b37fb11ec41af31e452e182f5e1604ee WatchSource:0}: Error finding container 11bd64d90c2f3c39528ff29557399c58b37fb11ec41af31e452e182f5e1604ee: Status 404 returned error can't find the container with id 11bd64d90c2f3c39528ff29557399c58b37fb11ec41af31e452e182f5e1604ee
Sep 09 13:08:53.096238 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:53.096078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq" event={"ID":"6807b4d1-145f-436f-8fcf-e88ce3e372e6","Type":"ContainerStarted","Data":"92b10bb8f7423001065496bf96d4f372846472dd9135c440f998e0f25a05dd17"}
Sep 09 13:08:53.096238 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:53.096140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq" event={"ID":"6807b4d1-145f-436f-8fcf-e88ce3e372e6","Type":"ContainerStarted","Data":"11bd64d90c2f3c39528ff29557399c58b37fb11ec41af31e452e182f5e1604ee"}
Sep 09 13:08:53.096745 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:53.096722    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:08:53.097861 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:53.097838    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws" event={"ID":"ba8fee37-6c76-4558-b530-4dae1aed42fe","Type":"ContainerStarted","Data":"a78cdaccfe38790c47e2b35fbbb48438eccda347d1dc67ecca42b793954a578e"}
Sep 09 13:08:53.130132 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:53.130084    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq" podStartSLOduration=1.130070118 podStartE2EDuration="1.130070118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:53.128953904 +0000 UTC m=+1219.129620002" watchObservedRunningTime="2026-09-09 13:08:53.130070118 +0000 UTC m=+1219.130736217"
Sep 09 13:08:54.104940 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:54.104896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws" event={"ID":"ba8fee37-6c76-4558-b530-4dae1aed42fe","Type":"ContainerStarted","Data":"f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8"}
Sep 09 13:08:54.105390 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:54.104966    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:08:54.123880 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:54.123816    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws" podStartSLOduration=2.776950095 podStartE2EDuration="3.123796042s" totalImagesPullingTime="346.845947ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:54.120737515 +0000 UTC m=+1220.121403617" watchObservedRunningTime="2026-09-09 13:08:54.123796042 +0000 UTC m=+1220.124462143"
Sep 09 13:08:55.047681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:55.047577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:08:56.061841 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:56.061793    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 13:08:56.062307 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:56.061857    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 13:08:58.127314 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:58.127207    2577 generic.go:356] "Generic (PLEG): container finished" podID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerID="3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e" exitCode=0
Sep 09 13:08:58.127314 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:58.127282    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" event={"ID":"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e","Type":"ContainerDied","Data":"3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e"}
Sep 09 13:08:58.127773 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:58.127715    2577 scope.go:117] "RemoveContainer" containerID="3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e"
Sep 09 13:08:58.128808 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:58.128774    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:58.128881 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:58.128867    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:58.128922 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:58.128903    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:08:59.134307 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:59.134270    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" event={"ID":"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e","Type":"ContainerStarted","Data":"52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83"}
Sep 09 13:08:59.840601 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:59.840561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:08:59.840825 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:59.840804    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 13:08:59.840875 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:08:59.840852    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 13:09:00.116430 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:00.116354    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:09:00.840091 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:00.840053    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 13:09:00.840509 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:00.840114    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 13:09:01.840995 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:01.840955    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 13:09:01.841392 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:01.841048    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 13:09:02.840427 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:02.840384    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 13:09:02.840615 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:02.840450    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 13:09:03.839985 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:03.839946    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 13:09:03.840480 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:03.840028    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 13:09:04.107620 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:04.107523    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:09:04.840824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:04.840790    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 09 13:09:04.841212 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:04.840860    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 09 13:09:05.843341 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:05.843309    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:09:05.843826 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:05.843562    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:09:05.844215 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:05.844200    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:09:32.617942 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.617909    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"]
Sep 09 13:09:32.620066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.620049    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.622376 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.622360    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-597fffff-openshift-default-dockercfg-6qlzj\""
Sep 09 13:09:32.631271 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.631249    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"]
Sep 09 13:09:32.719180 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.719143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-credential-socket\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.719466 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.719443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-socket\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.719599 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.719584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/394fed24-eb65-4103-831c-fd49232240fb-istio-podinfo\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.719765 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.719750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-istio-token\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.719900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.719886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-certs\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.720030 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.720014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2htl\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-kube-api-access-q2htl\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.720138 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.720120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-envoy\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.720248 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.720235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/394fed24-eb65-4103-831c-fd49232240fb-istiod-ca-cert\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.720375 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.720361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-data\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-istio-token\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-certs\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2htl\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-kube-api-access-q2htl\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-envoy\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/394fed24-eb65-4103-831c-fd49232240fb-istiod-ca-cert\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-data\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-credential-socket\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821683    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-socket\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.821750 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.821713    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/394fed24-eb65-4103-831c-fd49232240fb-istio-podinfo\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.822092 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.822065    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-credential-socket\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.822157 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.822138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-data\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.822215 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.822176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-certs\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.822353 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.822333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/394fed24-eb65-4103-831c-fd49232240fb-istiod-ca-cert\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.822457 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.822438    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-socket\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.824507 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.824481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-envoy\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.824852 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.824832    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/394fed24-eb65-4103-831c-fd49232240fb-istio-podinfo\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.829032 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.829011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-istio-token\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.829395 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.829375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2htl\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-kube-api-access-q2htl\") pod \"e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") " pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:32.930479 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:32.930455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:33.207158 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:33.207132    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"]
Sep 09 13:09:33.208056 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:09:33.208028    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394fed24_eb65_4103_831c_fd49232240fb.slice/crio-f74c78701afcb88977d31842bbcfa84aa2b9674bcaa0cb487b424cf26f159bba WatchSource:0}: Error finding container f74c78701afcb88977d31842bbcfa84aa2b9674bcaa0cb487b424cf26f159bba: Status 404 returned error can't find the container with id f74c78701afcb88977d31842bbcfa84aa2b9674bcaa0cb487b424cf26f159bba
Sep 09 13:09:33.210582 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:33.210545    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:09:33.210670 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:33.210629    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:09:33.210720 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:33.210693    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:09:33.300366 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:33.300333    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" event={"ID":"394fed24-eb65-4103-831c-fd49232240fb","Type":"ContainerStarted","Data":"9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2"}
Sep 09 13:09:33.300530 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:33.300400    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" event={"ID":"394fed24-eb65-4103-831c-fd49232240fb","Type":"ContainerStarted","Data":"f74c78701afcb88977d31842bbcfa84aa2b9674bcaa0cb487b424cf26f159bba"}
Sep 09 13:09:33.319397 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:33.319339    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" podStartSLOduration=1.319319581 podStartE2EDuration="1.319319581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:09:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:09:33.318410686 +0000 UTC m=+1259.319076801" watchObservedRunningTime="2026-09-09 13:09:33.319319581 +0000 UTC m=+1259.319985683"
Sep 09 13:09:33.931136 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:33.931105    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:34.936433 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:34.936402    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:35.310173 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:35.310075    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:35.310893 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:35.310873    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:37.146452 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.146423    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"]
Sep 09 13:09:37.149136 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.149116    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:37.154627 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.154607    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-597fffff\""
Sep 09 13:09:37.154765 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.154752    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-597fffff-dockercfg-jx8cs\""
Sep 09 13:09:37.160526 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.160502    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"]
Sep 09 13:09:37.264827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.264790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsf75\" (UniqueName: \"kubernetes.io/projected/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-kube-api-access-xsf75\") pod \"payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c\" (UID: \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\") " pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:37.265002 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.264941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-plugins-config-volume\") pod \"payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c\" (UID: \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\") " pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:37.366187 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.366145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-plugins-config-volume\") pod \"payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c\" (UID: \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\") " pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:37.366902 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.366877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsf75\" (UniqueName: \"kubernetes.io/projected/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-kube-api-access-xsf75\") pod \"payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c\" (UID: \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\") " pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:37.367346 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.367317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-plugins-config-volume\") pod \"payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c\" (UID: \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\") " pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:37.375743 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.375722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsf75\" (UniqueName: \"kubernetes.io/projected/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-kube-api-access-xsf75\") pod \"payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c\" (UID: \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\") " pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:37.462423 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.462334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:37.736626 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:37.736603    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"]
Sep 09 13:09:37.738289 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:09:37.738262    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f8c6a82_ede9_45e3_b7b1_95bfbe76dcd9.slice/crio-f8f73cfa63e97867f1f998ceb8c86e4c52f82570a1a206a99ea7b622f5faf234 WatchSource:0}: Error finding container f8f73cfa63e97867f1f998ceb8c86e4c52f82570a1a206a99ea7b622f5faf234: Status 404 returned error can't find the container with id f8f73cfa63e97867f1f998ceb8c86e4c52f82570a1a206a99ea7b622f5faf234
Sep 09 13:09:38.326713 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:38.326675    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c" event={"ID":"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9","Type":"ContainerStarted","Data":"00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb"}
Sep 09 13:09:38.327154 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:38.326735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c" event={"ID":"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9","Type":"ContainerStarted","Data":"f8f73cfa63e97867f1f998ceb8c86e4c52f82570a1a206a99ea7b622f5faf234"}
Sep 09 13:09:38.327154 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:38.326765    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:38.345071 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:38.345024    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c" podStartSLOduration=1.345009482 podStartE2EDuration="1.345009482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:09:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:09:38.342278538 +0000 UTC m=+1264.342944641" watchObservedRunningTime="2026-09-09 13:09:38.345009482 +0000 UTC m=+1264.345675581"
Sep 09 13:09:49.333894 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:49.333863    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:09:57.432416 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:57.432379    2577 generic.go:356] "Generic (PLEG): container finished" podID="394fed24-eb65-4103-831c-fd49232240fb" containerID="9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2" exitCode=0
Sep 09 13:09:57.432863 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:57.432449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" event={"ID":"394fed24-eb65-4103-831c-fd49232240fb","Type":"ContainerDied","Data":"9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2"}
Sep 09 13:09:57.432930 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:57.432916    2577 scope.go:117] "RemoveContainer" containerID="9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2"
Sep 09 13:09:57.434075 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:57.434046    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:09:57.434168 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:57.434116    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:09:57.434168 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:57.434149    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:09:58.439625 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:58.439590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" event={"ID":"394fed24-eb65-4103-831c-fd49232240fb","Type":"ContainerStarted","Data":"3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874"}
Sep 09 13:09:58.931521 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:58.931481    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:09:58.931766 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:58.931718    2577 patch_prober.go:28] interesting pod/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 09 13:09:58.931846 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:58.931778    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 09 13:09:59.931182 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:59.931147    2577 patch_prober.go:28] interesting pod/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 09 13:09:59.931553 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:09:59.931200    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 09 13:10:00.931807 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:00.931767    2577 patch_prober.go:28] interesting pod/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 09 13:10:00.932246 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:00.931831    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 09 13:10:01.931501 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:01.931462    2577 patch_prober.go:28] interesting pod/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 09 13:10:01.931731 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:01.931535    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 09 13:10:02.930989 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:02.930955    2577 patch_prober.go:28] interesting pod/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 09 13:10:02.931365 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:02.931021    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 09 13:10:03.930791 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:03.930757    2577 patch_prober.go:28] interesting pod/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 09 13:10:03.930971 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:03.930809    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 09 13:10:04.934955 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:04.934927    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:10:04.935308 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:04.935259    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:10:04.935974 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:10:04.935957    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:11:00.645388 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:00.645303    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"]
Sep 09 13:11:00.645982 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:00.645565    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c" podUID="5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9" containerName="payload-processing" containerID="cri-o://00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb" gracePeriod=30
Sep 09 13:11:00.895580 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:00.895516    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:11:00.965015 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:00.964987    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-plugins-config-volume\") pod \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\" (UID: \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\") "
Sep 09 13:11:00.965143 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:00.965042    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsf75\" (UniqueName: \"kubernetes.io/projected/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-kube-api-access-xsf75\") pod \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\" (UID: \"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9\") "
Sep 09 13:11:00.965358 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:00.965337    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9" (UID: "5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:11:00.967165 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:00.967149    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-kube-api-access-xsf75" (OuterVolumeSpecName: "kube-api-access-xsf75") pod "5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9" (UID: "5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9"). InnerVolumeSpecName "kube-api-access-xsf75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:11:01.066832 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.066791    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsf75\" (UniqueName: \"kubernetes.io/projected/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-kube-api-access-xsf75\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:11:01.066832 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.066823    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:11:01.749865 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.749827    2577 generic.go:356] "Generic (PLEG): container finished" podID="5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9" containerID="00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb" exitCode=0
Sep 09 13:11:01.750297 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.749870    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"
Sep 09 13:11:01.750297 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.749908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c" event={"ID":"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9","Type":"ContainerDied","Data":"00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb"}
Sep 09 13:11:01.750297 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.749957    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c" event={"ID":"5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9","Type":"ContainerDied","Data":"f8f73cfa63e97867f1f998ceb8c86e4c52f82570a1a206a99ea7b622f5faf234"}
Sep 09 13:11:01.750297 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.749979    2577 scope.go:117] "RemoveContainer" containerID="00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb"
Sep 09 13:11:01.759218 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.759203    2577 scope.go:117] "RemoveContainer" containerID="00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb"
Sep 09 13:11:01.759496 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:11:01.759477    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb\": container with ID starting with 00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb not found: ID does not exist" containerID="00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb"
Sep 09 13:11:01.759562 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.759509    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb"} err="failed to get container status \"00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb\": rpc error: code = NotFound desc = could not find container \"00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb\": container with ID starting with 00cc027b11ab3075517007c8d3480b65664456f0d01002c202e3222352dabdcb not found: ID does not exist"
Sep 09 13:11:01.770748 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.770723    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"]
Sep 09 13:11:01.774556 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:01.774535    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-597fffff-54c8c8c48f-24p5c"]
Sep 09 13:11:02.515515 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:11:02.515481    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9" path="/var/lib/kubelet/pods/5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9/volumes"
Sep 09 13:12:06.311459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:06.311421    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"]
Sep 09 13:12:06.312087 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:06.311688    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy" containerID="cri-o://3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874" gracePeriod=30
Sep 09 13:12:11.554235 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.554207    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:12:11.706902 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.706874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-certs\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707067 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.706925    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-socket\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707067 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.706968    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-istio-token\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707067 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.706999    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-credential-socket\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707067 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707049    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-data\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707261 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707073    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2htl\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-kube-api-access-q2htl\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707261 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707124    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:12:11.707261 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707131    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-envoy\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707261 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707186    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/394fed24-eb65-4103-831c-fd49232240fb-istiod-ca-cert\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707469 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707269    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/394fed24-eb65-4103-831c-fd49232240fb-istio-podinfo\") pod \"394fed24-eb65-4103-831c-fd49232240fb\" (UID: \"394fed24-eb65-4103-831c-fd49232240fb\") "
Sep 09 13:12:11.707469 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707271    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:12:11.707469 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707361    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:12:11.707469 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707394    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:12:11.707673 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707592    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/394fed24-eb65-4103-831c-fd49232240fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:12:11.707935 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707876    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:11.707935 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707898    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:11.707935 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707914    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:11.707935 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707927    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:11.707935 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.707940    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/394fed24-eb65-4103-831c-fd49232240fb-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:11.709151 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.709119    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:12:11.709242 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.709181    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:12:11.709309 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.709286    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/394fed24-eb65-4103-831c-fd49232240fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:12:11.709502 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.709481    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-kube-api-access-q2htl" (OuterVolumeSpecName: "kube-api-access-q2htl") pod "394fed24-eb65-4103-831c-fd49232240fb" (UID: "394fed24-eb65-4103-831c-fd49232240fb"). InnerVolumeSpecName "kube-api-access-q2htl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:12:11.809092 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.809054    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:11.809092 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.809086    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2htl\" (UniqueName: \"kubernetes.io/projected/394fed24-eb65-4103-831c-fd49232240fb-kube-api-access-q2htl\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:11.809092 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.809098    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/394fed24-eb65-4103-831c-fd49232240fb-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:11.809334 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:11.809108    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/394fed24-eb65-4103-831c-fd49232240fb-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:12:12.102782 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.102686    2577 generic.go:356] "Generic (PLEG): container finished" podID="394fed24-eb65-4103-831c-fd49232240fb" containerID="3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874" exitCode=0
Sep 09 13:12:12.102782 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.102740    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"
Sep 09 13:12:12.102782 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.102770    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" event={"ID":"394fed24-eb65-4103-831c-fd49232240fb","Type":"ContainerDied","Data":"3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874"}
Sep 09 13:12:12.103035 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.102828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw" event={"ID":"394fed24-eb65-4103-831c-fd49232240fb","Type":"ContainerDied","Data":"f74c78701afcb88977d31842bbcfa84aa2b9674bcaa0cb487b424cf26f159bba"}
Sep 09 13:12:12.103035 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.102851    2577 scope.go:117] "RemoveContainer" containerID="3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874"
Sep 09 13:12:12.112348 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.112331    2577 scope.go:117] "RemoveContainer" containerID="9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2"
Sep 09 13:12:12.120570 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.120554    2577 scope.go:117] "RemoveContainer" containerID="3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874"
Sep 09 13:12:12.120859 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:12:12.120840    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874\": container with ID starting with 3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874 not found: ID does not exist" containerID="3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874"
Sep 09 13:12:12.120952 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.120872    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874"} err="failed to get container status \"3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874\": rpc error: code = NotFound desc = could not find container \"3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874\": container with ID starting with 3790498d0826abe9f15d4222bd48a77b1050c9a62bf189e02a50b220466cd874 not found: ID does not exist"
Sep 09 13:12:12.120952 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.120895    2577 scope.go:117] "RemoveContainer" containerID="9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2"
Sep 09 13:12:12.121159 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:12:12.121142    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2\": container with ID starting with 9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2 not found: ID does not exist" containerID="9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2"
Sep 09 13:12:12.121198 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.121165    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2"} err="failed to get container status \"9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2\": rpc error: code = NotFound desc = could not find container \"9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2\": container with ID starting with 9304c3e67fc8205c2c1c719a052dacd3f6fcc57412c50332209bdf99d6d22ec2 not found: ID does not exist"
Sep 09 13:12:12.128713 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.128693    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"]
Sep 09 13:12:12.133326 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.133305    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-597fffff-openshift-default-7b6bc79484-cjccw"]
Sep 09 13:12:12.515324 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:12:12.515284    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="394fed24-eb65-4103-831c-fd49232240fb" path="/var/lib/kubelet/pods/394fed24-eb65-4103-831c-fd49232240fb/volumes"
Sep 09 13:13:00.903666 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.903617    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"]
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.903994    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9" containerName="payload-processing"
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.904007    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9" containerName="payload-processing"
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.904016    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy"
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.904022    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy"
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.904038    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy"
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.904043    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy"
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.904105    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f8c6a82-ede9-45e3-b7b1-95bfbe76dcd9" containerName="payload-processing"
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.904114    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy"
Sep 09 13:13:00.904125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.904121    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="394fed24-eb65-4103-831c-fd49232240fb" containerName="istio-proxy"
Sep 09 13:13:00.908341 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.908317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:00.910812 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.910792    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-21da34-openshift-default-dockercfg-vpcfw\""
Sep 09 13:13:00.921954 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:00.921932    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"]
Sep 09 13:13:01.051570 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-certs\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.051773 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-credential-socket\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.051773 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-data\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.051773 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-socket\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.051773 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051772    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a243bbb9-157f-474a-a454-54328a457dc2-istiod-ca-cert\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.051939 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-envoy\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.051939 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a243bbb9-157f-474a-a454-54328a457dc2-istio-podinfo\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.051939 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-istio-token\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.051939 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.051893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhlcq\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-kube-api-access-rhlcq\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.152864 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.152823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-credential-socket\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.152864 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.152865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-data\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153142 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.152900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-socket\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153142 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.152930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a243bbb9-157f-474a-a454-54328a457dc2-istiod-ca-cert\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153142 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.152946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-envoy\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153142 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.152973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a243bbb9-157f-474a-a454-54328a457dc2-istio-podinfo\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153142 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.152992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-istio-token\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153142 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.153010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhlcq\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-kube-api-access-rhlcq\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153142 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.153061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-certs\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153546 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.153292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-credential-socket\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153841 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.153781    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-data\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153946 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.153836    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a243bbb9-157f-474a-a454-54328a457dc2-istiod-ca-cert\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153946 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.153858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-socket\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.153946 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.153898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-certs\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.155519 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.155500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-envoy\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.155681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.155659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a243bbb9-157f-474a-a454-54328a457dc2-istio-podinfo\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.160905 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.160872    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-istio-token\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.160905 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.160895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhlcq\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-kube-api-access-rhlcq\") pod \"e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") " pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.219531 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.219495    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:01.498476 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.498441    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"]
Sep 09 13:13:01.500087 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:13:01.500059    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda243bbb9_157f_474a_a454_54328a457dc2.slice/crio-ff4cb0ef2ef7a7cb78696926b53727e5b41c671e9f7aec004ed8cedefb8acbba WatchSource:0}: Error finding container ff4cb0ef2ef7a7cb78696926b53727e5b41c671e9f7aec004ed8cedefb8acbba: Status 404 returned error can't find the container with id ff4cb0ef2ef7a7cb78696926b53727e5b41c671e9f7aec004ed8cedefb8acbba
Sep 09 13:13:01.502487 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.502454    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:13:01.502567 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.502521    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:13:01.502567 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:01.502553    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:13:02.362277 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:02.362245    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" event={"ID":"a243bbb9-157f-474a-a454-54328a457dc2","Type":"ContainerStarted","Data":"8eb71af83d15080358d2967e23fd306bd550b733a5d8cc795870741e0e7c7d19"}
Sep 09 13:13:02.362709 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:02.362320    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" event={"ID":"a243bbb9-157f-474a-a454-54328a457dc2","Type":"ContainerStarted","Data":"ff4cb0ef2ef7a7cb78696926b53727e5b41c671e9f7aec004ed8cedefb8acbba"}
Sep 09 13:13:02.382009 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:02.381951    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" podStartSLOduration=2.381932296 podStartE2EDuration="2.381932296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:13:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:13:02.380858755 +0000 UTC m=+1468.381524859" watchObservedRunningTime="2026-09-09 13:13:02.381932296 +0000 UTC m=+1468.382598397"
Sep 09 13:13:03.220540 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:03.220500    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:03.225690 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:03.225667    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:03.367890 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:03.367856    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:03.368963 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:03.368941    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:06.382473 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.382439    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"]
Sep 09 13:13:06.385733 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.385714    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:06.388175 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.388150    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-21da34\""
Sep 09 13:13:06.388286 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.388162    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-21da34-dockercfg-2wv4m\""
Sep 09 13:13:06.397482 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.397459    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"]
Sep 09 13:13:06.402818 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.402793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6bb8\" (UniqueName: \"kubernetes.io/projected/675f66b4-255b-4efb-95f5-194e888d09d1-kube-api-access-j6bb8\") pod \"payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz\" (UID: \"675f66b4-255b-4efb-95f5-194e888d09d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:06.402944 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.402926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/675f66b4-255b-4efb-95f5-194e888d09d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz\" (UID: \"675f66b4-255b-4efb-95f5-194e888d09d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:06.412850 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.412824    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"]
Sep 09 13:13:06.416711 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.416691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:06.432778 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.432754    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"]
Sep 09 13:13:06.506396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.505132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20536997-e4b5-46c7-b087-4aa56332afd7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-21da34-b574c65-bskjl\" (UID: \"20536997-e4b5-46c7-b087-4aa56332afd7\") " pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:06.506396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.505276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs8zl\" (UniqueName: \"kubernetes.io/projected/20536997-e4b5-46c7-b087-4aa56332afd7-kube-api-access-rs8zl\") pod \"payload-processing-e2e-api-a-21da34-b574c65-bskjl\" (UID: \"20536997-e4b5-46c7-b087-4aa56332afd7\") " pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:06.506396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.505320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/675f66b4-255b-4efb-95f5-194e888d09d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz\" (UID: \"675f66b4-255b-4efb-95f5-194e888d09d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:06.506396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.505350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6bb8\" (UniqueName: \"kubernetes.io/projected/675f66b4-255b-4efb-95f5-194e888d09d1-kube-api-access-j6bb8\") pod \"payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz\" (UID: \"675f66b4-255b-4efb-95f5-194e888d09d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:06.506813 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.506535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/675f66b4-255b-4efb-95f5-194e888d09d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz\" (UID: \"675f66b4-255b-4efb-95f5-194e888d09d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:06.517256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.517216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6bb8\" (UniqueName: \"kubernetes.io/projected/675f66b4-255b-4efb-95f5-194e888d09d1-kube-api-access-j6bb8\") pod \"payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz\" (UID: \"675f66b4-255b-4efb-95f5-194e888d09d1\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:06.606460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.606421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs8zl\" (UniqueName: \"kubernetes.io/projected/20536997-e4b5-46c7-b087-4aa56332afd7-kube-api-access-rs8zl\") pod \"payload-processing-e2e-api-a-21da34-b574c65-bskjl\" (UID: \"20536997-e4b5-46c7-b087-4aa56332afd7\") " pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:06.606694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.606594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20536997-e4b5-46c7-b087-4aa56332afd7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-21da34-b574c65-bskjl\" (UID: \"20536997-e4b5-46c7-b087-4aa56332afd7\") " pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:06.607317 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.607292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20536997-e4b5-46c7-b087-4aa56332afd7-plugins-config-volume\") pod \"payload-processing-e2e-api-a-21da34-b574c65-bskjl\" (UID: \"20536997-e4b5-46c7-b087-4aa56332afd7\") " pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:06.614562 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.614543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs8zl\" (UniqueName: \"kubernetes.io/projected/20536997-e4b5-46c7-b087-4aa56332afd7-kube-api-access-rs8zl\") pod \"payload-processing-e2e-api-a-21da34-b574c65-bskjl\" (UID: \"20536997-e4b5-46c7-b087-4aa56332afd7\") " pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:06.697529 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.697437    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:06.726801 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.726770    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:06.969882 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:06.969857    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"]
Sep 09 13:13:06.971811 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:13:06.971779    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod675f66b4_255b_4efb_95f5_194e888d09d1.slice/crio-605ccedd936256f5c1e5e9aa369070ea887b94f80ba29811a268e42f4ea0382f WatchSource:0}: Error finding container 605ccedd936256f5c1e5e9aa369070ea887b94f80ba29811a268e42f4ea0382f: Status 404 returned error can't find the container with id 605ccedd936256f5c1e5e9aa369070ea887b94f80ba29811a268e42f4ea0382f
Sep 09 13:13:07.008089 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.008067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"]
Sep 09 13:13:07.010487 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:13:07.010457    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20536997_e4b5_46c7_b087_4aa56332afd7.slice/crio-b9e1918c2261016d18a8c3ec2380a3be66cbb31f4a38f1e10f374c703dfe0b03 WatchSource:0}: Error finding container b9e1918c2261016d18a8c3ec2380a3be66cbb31f4a38f1e10f374c703dfe0b03: Status 404 returned error can't find the container with id b9e1918c2261016d18a8c3ec2380a3be66cbb31f4a38f1e10f374c703dfe0b03
Sep 09 13:13:07.388278 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.388242    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl" event={"ID":"20536997-e4b5-46c7-b087-4aa56332afd7","Type":"ContainerStarted","Data":"091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8"}
Sep 09 13:13:07.388802 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.388301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl" event={"ID":"20536997-e4b5-46c7-b087-4aa56332afd7","Type":"ContainerStarted","Data":"b9e1918c2261016d18a8c3ec2380a3be66cbb31f4a38f1e10f374c703dfe0b03"}
Sep 09 13:13:07.388802 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.388419    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:07.389971 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.389945    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz" event={"ID":"675f66b4-255b-4efb-95f5-194e888d09d1","Type":"ContainerStarted","Data":"b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105"}
Sep 09 13:13:07.390117 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.389990    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz" event={"ID":"675f66b4-255b-4efb-95f5-194e888d09d1","Type":"ContainerStarted","Data":"605ccedd936256f5c1e5e9aa369070ea887b94f80ba29811a268e42f4ea0382f"}
Sep 09 13:13:07.390117 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.390042    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:07.403936 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.403884    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl" podStartSLOduration=1.4038628229999999 podStartE2EDuration="1.403862823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:13:07.402545138 +0000 UTC m=+1473.403211238" watchObservedRunningTime="2026-09-09 13:13:07.403862823 +0000 UTC m=+1473.404528927"
Sep 09 13:13:07.426931 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:07.426877    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz" podStartSLOduration=1.42686199 podStartE2EDuration="1.42686199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:13:07.425251338 +0000 UTC m=+1473.425917429" watchObservedRunningTime="2026-09-09 13:13:07.42686199 +0000 UTC m=+1473.427528090"
Sep 09 13:13:18.398417 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:18.398382    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:13:18.398904 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:18.398566    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:13:25.494860 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:25.494826    2577 generic.go:356] "Generic (PLEG): container finished" podID="a243bbb9-157f-474a-a454-54328a457dc2" containerID="8eb71af83d15080358d2967e23fd306bd550b733a5d8cc795870741e0e7c7d19" exitCode=0
Sep 09 13:13:25.495322 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:25.494899    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" event={"ID":"a243bbb9-157f-474a-a454-54328a457dc2","Type":"ContainerDied","Data":"8eb71af83d15080358d2967e23fd306bd550b733a5d8cc795870741e0e7c7d19"}
Sep 09 13:13:25.495371 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:25.495340    2577 scope.go:117] "RemoveContainer" containerID="8eb71af83d15080358d2967e23fd306bd550b733a5d8cc795870741e0e7c7d19"
Sep 09 13:13:25.496550 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:25.496527    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:13:25.496602 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:25.496591    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:13:25.496652 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:25.496621    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:13:26.502862 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:26.502825    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" event={"ID":"a243bbb9-157f-474a-a454-54328a457dc2","Type":"ContainerStarted","Data":"f641b153ac082eed5db791dc223a1af95c4c171b47795d9500f6943ce3546d82"}
Sep 09 13:13:27.219893 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:27.219852    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:27.220111 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:27.220093    2577 patch_prober.go:28] interesting pod/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 09 13:13:27.220162 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:27.220136    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 09 13:13:28.220389 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:28.220355    2577 patch_prober.go:28] interesting pod/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 09 13:13:28.220801 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:28.220412    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 09 13:13:29.220818 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:29.220776    2577 patch_prober.go:28] interesting pod/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 09 13:13:29.221211 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:29.220853    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 09 13:13:30.220526 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:30.220489    2577 patch_prober.go:28] interesting pod/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 09 13:13:30.220755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:30.220572    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 09 13:13:31.220554 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:31.220514    2577 patch_prober.go:28] interesting pod/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Sep 09 13:13:31.220983 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:31.220576    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Sep 09 13:13:32.224008 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:32.223979    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:32.224364 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:32.224254    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:13:32.224961 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:13:32.224943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:14:18.045561 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.045522    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"]
Sep 09 13:14:18.046210 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.045803    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl" podUID="20536997-e4b5-46c7-b087-4aa56332afd7" containerName="payload-processing" containerID="cri-o://091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8" gracePeriod=30
Sep 09 13:14:18.052180 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.052145    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"]
Sep 09 13:14:18.052429 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.052380    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz" podUID="675f66b4-255b-4efb-95f5-194e888d09d1" containerName="payload-pre-processing" containerID="cri-o://b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105" gracePeriod=30
Sep 09 13:14:18.333853 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.333822    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:14:18.336719 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.336692    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:14:18.499849 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.499817    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20536997-e4b5-46c7-b087-4aa56332afd7-plugins-config-volume\") pod \"20536997-e4b5-46c7-b087-4aa56332afd7\" (UID: \"20536997-e4b5-46c7-b087-4aa56332afd7\") "
Sep 09 13:14:18.500073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.499877    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rs8zl\" (UniqueName: \"kubernetes.io/projected/20536997-e4b5-46c7-b087-4aa56332afd7-kube-api-access-rs8zl\") pod \"20536997-e4b5-46c7-b087-4aa56332afd7\" (UID: \"20536997-e4b5-46c7-b087-4aa56332afd7\") "
Sep 09 13:14:18.500073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.499950    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/675f66b4-255b-4efb-95f5-194e888d09d1-plugins-config-volume\") pod \"675f66b4-255b-4efb-95f5-194e888d09d1\" (UID: \"675f66b4-255b-4efb-95f5-194e888d09d1\") "
Sep 09 13:14:18.500073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.500005    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6bb8\" (UniqueName: \"kubernetes.io/projected/675f66b4-255b-4efb-95f5-194e888d09d1-kube-api-access-j6bb8\") pod \"675f66b4-255b-4efb-95f5-194e888d09d1\" (UID: \"675f66b4-255b-4efb-95f5-194e888d09d1\") "
Sep 09 13:14:18.500310 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.500280    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20536997-e4b5-46c7-b087-4aa56332afd7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20536997-e4b5-46c7-b087-4aa56332afd7" (UID: "20536997-e4b5-46c7-b087-4aa56332afd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:18.500367 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.500324    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/675f66b4-255b-4efb-95f5-194e888d09d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "675f66b4-255b-4efb-95f5-194e888d09d1" (UID: "675f66b4-255b-4efb-95f5-194e888d09d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:18.502215 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.502192    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20536997-e4b5-46c7-b087-4aa56332afd7-kube-api-access-rs8zl" (OuterVolumeSpecName: "kube-api-access-rs8zl") pod "20536997-e4b5-46c7-b087-4aa56332afd7" (UID: "20536997-e4b5-46c7-b087-4aa56332afd7"). InnerVolumeSpecName "kube-api-access-rs8zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:14:18.502338 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.502317    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/675f66b4-255b-4efb-95f5-194e888d09d1-kube-api-access-j6bb8" (OuterVolumeSpecName: "kube-api-access-j6bb8") pod "675f66b4-255b-4efb-95f5-194e888d09d1" (UID: "675f66b4-255b-4efb-95f5-194e888d09d1"). InnerVolumeSpecName "kube-api-access-j6bb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:14:18.601904 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.601819    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6bb8\" (UniqueName: \"kubernetes.io/projected/675f66b4-255b-4efb-95f5-194e888d09d1-kube-api-access-j6bb8\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:18.601904 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.601851    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20536997-e4b5-46c7-b087-4aa56332afd7-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:18.601904 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.601863    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs8zl\" (UniqueName: \"kubernetes.io/projected/20536997-e4b5-46c7-b087-4aa56332afd7-kube-api-access-rs8zl\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:18.601904 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.601875    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/675f66b4-255b-4efb-95f5-194e888d09d1-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:18.767344 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.767307    2577 generic.go:356] "Generic (PLEG): container finished" podID="20536997-e4b5-46c7-b087-4aa56332afd7" containerID="091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8" exitCode=0
Sep 09 13:14:18.767490 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.767351    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"
Sep 09 13:14:18.767490 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.767391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl" event={"ID":"20536997-e4b5-46c7-b087-4aa56332afd7","Type":"ContainerDied","Data":"091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8"}
Sep 09 13:14:18.767490 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.767463    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl" event={"ID":"20536997-e4b5-46c7-b087-4aa56332afd7","Type":"ContainerDied","Data":"b9e1918c2261016d18a8c3ec2380a3be66cbb31f4a38f1e10f374c703dfe0b03"}
Sep 09 13:14:18.767666 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.767504    2577 scope.go:117] "RemoveContainer" containerID="091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8"
Sep 09 13:14:18.768830 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.768804    2577 generic.go:356] "Generic (PLEG): container finished" podID="675f66b4-255b-4efb-95f5-194e888d09d1" containerID="b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105" exitCode=0
Sep 09 13:14:18.768938 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.768854    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"
Sep 09 13:14:18.768938 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.768920    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz" event={"ID":"675f66b4-255b-4efb-95f5-194e888d09d1","Type":"ContainerDied","Data":"b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105"}
Sep 09 13:14:18.769048 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.768949    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz" event={"ID":"675f66b4-255b-4efb-95f5-194e888d09d1","Type":"ContainerDied","Data":"605ccedd936256f5c1e5e9aa369070ea887b94f80ba29811a268e42f4ea0382f"}
Sep 09 13:14:18.776556 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.776540    2577 scope.go:117] "RemoveContainer" containerID="091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8"
Sep 09 13:14:18.776843 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:14:18.776826    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8\": container with ID starting with 091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8 not found: ID does not exist" containerID="091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8"
Sep 09 13:14:18.776903 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.776852    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8"} err="failed to get container status \"091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8\": rpc error: code = NotFound desc = could not find container \"091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8\": container with ID starting with 091ba92465ee335fd430570d5879ac5e8362c885b50b8f99261133effc84d9d8 not found: ID does not exist"
Sep 09 13:14:18.776903 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.776868    2577 scope.go:117] "RemoveContainer" containerID="b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105"
Sep 09 13:14:18.785295 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.785276    2577 scope.go:117] "RemoveContainer" containerID="b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105"
Sep 09 13:14:18.785588 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:14:18.785563    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105\": container with ID starting with b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105 not found: ID does not exist" containerID="b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105"
Sep 09 13:14:18.785658 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.785599    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105"} err="failed to get container status \"b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105\": rpc error: code = NotFound desc = could not find container \"b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105\": container with ID starting with b800408d1296ea39c852dc1f56fd2d7af398cc483c9577ba79f82d5c04f9c105 not found: ID does not exist"
Sep 09 13:14:18.785658 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.785611    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"]
Sep 09 13:14:18.789000 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.788980    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-21da34-5ffbbb5c9d-h5rbz"]
Sep 09 13:14:18.797722 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.797705    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"]
Sep 09 13:14:18.801753 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:18.801737    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-21da34-b574c65-bskjl"]
Sep 09 13:14:20.515330 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:20.515291    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20536997-e4b5-46c7-b087-4aa56332afd7" path="/var/lib/kubelet/pods/20536997-e4b5-46c7-b087-4aa56332afd7/volumes"
Sep 09 13:14:20.515730 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:20.515672    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="675f66b4-255b-4efb-95f5-194e888d09d1" path="/var/lib/kubelet/pods/675f66b4-255b-4efb-95f5-194e888d09d1/volumes"
Sep 09 13:14:24.601789 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:24.601759    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"]
Sep 09 13:14:24.602380 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:24.601990    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy" containerID="cri-o://f641b153ac082eed5db791dc223a1af95c4c171b47795d9500f6943ce3546d82" gracePeriod=30
Sep 09 13:14:29.827754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.827718    2577 generic.go:356] "Generic (PLEG): container finished" podID="a243bbb9-157f-474a-a454-54328a457dc2" containerID="f641b153ac082eed5db791dc223a1af95c4c171b47795d9500f6943ce3546d82" exitCode=0
Sep 09 13:14:29.828171 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.827756    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" event={"ID":"a243bbb9-157f-474a-a454-54328a457dc2","Type":"ContainerDied","Data":"f641b153ac082eed5db791dc223a1af95c4c171b47795d9500f6943ce3546d82"}
Sep 09 13:14:29.828171 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.827821    2577 scope.go:117] "RemoveContainer" containerID="8eb71af83d15080358d2967e23fd306bd550b733a5d8cc795870741e0e7c7d19"
Sep 09 13:14:29.863114 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.863036    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:14:29.904204 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904146    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a243bbb9-157f-474a-a454-54328a457dc2-istiod-ca-cert\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904204 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904202    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-istio-token\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904470 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904257    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-certs\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904470 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904282    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-socket\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904470 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904321    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a243bbb9-157f-474a-a454-54328a457dc2-istio-podinfo\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904470 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904344    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhlcq\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-kube-api-access-rhlcq\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904598 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904477    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-envoy\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904598 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904518    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-data\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904598 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904570    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-credential-socket\") pod \"a243bbb9-157f-474a-a454-54328a457dc2\" (UID: \"a243bbb9-157f-474a-a454-54328a457dc2\") "
Sep 09 13:14:29.904728 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904609    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a243bbb9-157f-474a-a454-54328a457dc2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:29.904728 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904630    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:29.904982 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.904841    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-data" (OuterVolumeSpecName: "istio-data") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:29.905077 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.905038    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a243bbb9-157f-474a-a454-54328a457dc2-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:29.905077 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.905043    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:29.905077 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.905059    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:29.905397 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.905093    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:29.905397 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.905110    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:29.906862 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.906835    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-istio-token" (OuterVolumeSpecName: "istio-token") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:14:29.906972 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.906943    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:29.907783 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.907760    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-kube-api-access-rhlcq" (OuterVolumeSpecName: "kube-api-access-rhlcq") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "kube-api-access-rhlcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:14:29.907835 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:29.907771    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a243bbb9-157f-474a-a454-54328a457dc2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a243bbb9-157f-474a-a454-54328a457dc2" (UID: "a243bbb9-157f-474a-a454-54328a457dc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:14:30.006273 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.006225    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:30.006273 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.006265    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:30.006273 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.006275    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a243bbb9-157f-474a-a454-54328a457dc2-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:30.006273 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.006283    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhlcq\" (UniqueName: \"kubernetes.io/projected/a243bbb9-157f-474a-a454-54328a457dc2-kube-api-access-rhlcq\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:30.006568 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.006293    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:30.006568 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.006301    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a243bbb9-157f-474a-a454-54328a457dc2-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:30.834523 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.834440    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"
Sep 09 13:14:30.834933 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.834440    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h" event={"ID":"a243bbb9-157f-474a-a454-54328a457dc2","Type":"ContainerDied","Data":"ff4cb0ef2ef7a7cb78696926b53727e5b41c671e9f7aec004ed8cedefb8acbba"}
Sep 09 13:14:30.834933 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.834577    2577 scope.go:117] "RemoveContainer" containerID="f641b153ac082eed5db791dc223a1af95c4c171b47795d9500f6943ce3546d82"
Sep 09 13:14:30.852691 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.852662    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"]
Sep 09 13:14:30.856328 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:30.856304    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-21da34-openshift-default-6f8b9c6f58-km77h"]
Sep 09 13:14:32.515336 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:32.515300    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a243bbb9-157f-474a-a454-54328a457dc2" path="/var/lib/kubelet/pods/a243bbb9-157f-474a-a454-54328a457dc2/volumes"
Sep 09 13:14:34.628476 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:34.628449    2577 scope.go:117] "RemoveContainer" containerID="ef1b99f39e8ab91969d6455b687bb244d42d6e127e0e26cb7d4b172fb0692e23"
Sep 09 13:14:34.636657 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:34.636622    2577 scope.go:117] "RemoveContainer" containerID="12b148d2288cc7aeb8a4e79654a15922ede954f96a21a0b9aafc2928262b3c53"
Sep 09 13:14:50.718443 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.718405    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"]
Sep 09 13:14:50.719009 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.718986    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy"
Sep 09 13:14:50.719119 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719013    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy"
Sep 09 13:14:50.719119 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719041    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20536997-e4b5-46c7-b087-4aa56332afd7" containerName="payload-processing"
Sep 09 13:14:50.719119 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719049    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="20536997-e4b5-46c7-b087-4aa56332afd7" containerName="payload-processing"
Sep 09 13:14:50.719119 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719061    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="675f66b4-255b-4efb-95f5-194e888d09d1" containerName="payload-pre-processing"
Sep 09 13:14:50.719119 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719070    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="675f66b4-255b-4efb-95f5-194e888d09d1" containerName="payload-pre-processing"
Sep 09 13:14:50.719357 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719173    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy"
Sep 09 13:14:50.719357 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719185    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy"
Sep 09 13:14:50.719357 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719198    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="20536997-e4b5-46c7-b087-4aa56332afd7" containerName="payload-processing"
Sep 09 13:14:50.719357 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.719208    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="675f66b4-255b-4efb-95f5-194e888d09d1" containerName="payload-pre-processing"
Sep 09 13:14:50.723883 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.723864    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.726350 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.726329    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-458b0f-openshift-default-dockercfg-qcrjr\""
Sep 09 13:14:50.732403 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.732380    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"]
Sep 09 13:14:50.801624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-credential-socket\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.801818 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801660    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-data\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.801818 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b8304a-7f81-40f8-aa68-de9b171547f8-istiod-ca-cert\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.801818 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-podinfo\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.801818 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2t4b\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-kube-api-access-s2t4b\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.802044 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-socket\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.802044 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-token\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.802044 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-envoy\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.802044 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.801960    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-certs\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903227 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-credential-socket\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903445 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-data\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903445 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b8304a-7f81-40f8-aa68-de9b171547f8-istiod-ca-cert\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903445 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-podinfo\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903629 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2t4b\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-kube-api-access-s2t4b\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903629 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-socket\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903629 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-credential-socket\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903690    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-data\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-token\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-envoy\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-certs\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.903994 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.903878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-socket\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.904061 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.904043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b8304a-7f81-40f8-aa68-de9b171547f8-istiod-ca-cert\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.904153 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.904134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-certs\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.906036 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.906011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-podinfo\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.906103 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.906052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-envoy\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.911354 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.911333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-token\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:50.911899 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:50.911878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2t4b\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-kube-api-access-s2t4b\") pod \"e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") " pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:51.036483 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:51.036380    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:51.320729 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:51.320706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"]
Sep 09 13:14:51.322860 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:14:51.322832    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b8304a_7f81_40f8_aa68_de9b171547f8.slice/crio-c3b94ff9723b094798bc64843c6d57ab47844c870038e51455e6e71ffc7e568f WatchSource:0}: Error finding container c3b94ff9723b094798bc64843c6d57ab47844c870038e51455e6e71ffc7e568f: Status 404 returned error can't find the container with id c3b94ff9723b094798bc64843c6d57ab47844c870038e51455e6e71ffc7e568f
Sep 09 13:14:51.328671 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:51.325604    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:14:51.328671 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:51.325815    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:14:51.328671 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:51.325905    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:14:51.942886 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:51.942850    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" event={"ID":"76b8304a-7f81-40f8-aa68-de9b171547f8","Type":"ContainerStarted","Data":"a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800"}
Sep 09 13:14:51.943303 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:51.942929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" event={"ID":"76b8304a-7f81-40f8-aa68-de9b171547f8","Type":"ContainerStarted","Data":"c3b94ff9723b094798bc64843c6d57ab47844c870038e51455e6e71ffc7e568f"}
Sep 09 13:14:51.962662 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:51.962585    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" podStartSLOduration=1.962565004 podStartE2EDuration="1.962565004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:14:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:14:51.962115711 +0000 UTC m=+1577.962781812" watchObservedRunningTime="2026-09-09 13:14:51.962565004 +0000 UTC m=+1577.963231105"
Sep 09 13:14:52.037382 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:52.037346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:52.042816 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:52.042789    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:52.948007 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:52.947969    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:52.948965 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:52.948943    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:14:55.809755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.809724    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"]
Sep 09 13:14:55.810183 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.810092    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy"
Sep 09 13:14:55.810183 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.810104    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a243bbb9-157f-474a-a454-54328a457dc2" containerName="istio-proxy"
Sep 09 13:14:55.813023 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.813007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:55.815496 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.815473    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-458b0f\""
Sep 09 13:14:55.815767 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.815752    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-458b0f-dockercfg-kd4k5\""
Sep 09 13:14:55.824178 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.824156    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"]
Sep 09 13:14:55.949146 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.949111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqdh5\" (UniqueName: \"kubernetes.io/projected/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-kube-api-access-xqdh5\") pod \"payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q\" (UID: \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:55.949354 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:55.949308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q\" (UID: \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:56.050369 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.050333    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqdh5\" (UniqueName: \"kubernetes.io/projected/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-kube-api-access-xqdh5\") pod \"payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q\" (UID: \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:56.050544 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.050401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q\" (UID: \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:56.051074 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.051056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q\" (UID: \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:56.058231 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.058203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqdh5\" (UniqueName: \"kubernetes.io/projected/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-kube-api-access-xqdh5\") pod \"payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q\" (UID: \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:56.124261 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.124197    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:56.392753 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.392730    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"]
Sep 09 13:14:56.394435 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:14:56.394411    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0fa2864_cd6d_45d9_9829_89aa102d4e8e.slice/crio-9b994cbba02b6a984ef3de29f26f79a84761e1f0b01e7a107cabcd13d9128cfc WatchSource:0}: Error finding container 9b994cbba02b6a984ef3de29f26f79a84761e1f0b01e7a107cabcd13d9128cfc: Status 404 returned error can't find the container with id 9b994cbba02b6a984ef3de29f26f79a84761e1f0b01e7a107cabcd13d9128cfc
Sep 09 13:14:56.969406 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.969369    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q" event={"ID":"a0fa2864-cd6d-45d9-9829-89aa102d4e8e","Type":"ContainerStarted","Data":"f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c"}
Sep 09 13:14:56.969829 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.969439    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q" event={"ID":"a0fa2864-cd6d-45d9-9829-89aa102d4e8e","Type":"ContainerStarted","Data":"9b994cbba02b6a984ef3de29f26f79a84761e1f0b01e7a107cabcd13d9128cfc"}
Sep 09 13:14:56.969829 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.969469    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:14:56.984916 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:56.984863    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q" podStartSLOduration=1.9848449769999998 podStartE2EDuration="1.984844977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:14:56.984466293 +0000 UTC m=+1582.985132393" watchObservedRunningTime="2026-09-09 13:14:56.984844977 +0000 UTC m=+1582.985511077"
Sep 09 13:14:58.081413 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.081380    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"]
Sep 09 13:14:58.081895 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.081623    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b" podUID="9a42376a-140f-4b99-a2d2-e218ead49187" containerName="maas-api" containerID="cri-o://81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565" gracePeriod=30
Sep 09 13:14:58.357508 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.357485    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:14:58.472269 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.472240    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls\") pod \"9a42376a-140f-4b99-a2d2-e218ead49187\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") "
Sep 09 13:14:58.472459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.472335    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs4fq\" (UniqueName: \"kubernetes.io/projected/9a42376a-140f-4b99-a2d2-e218ead49187-kube-api-access-gs4fq\") pod \"9a42376a-140f-4b99-a2d2-e218ead49187\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") "
Sep 09 13:14:58.472459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.472405    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a42376a-140f-4b99-a2d2-e218ead49187-openshift-service-ca\") pod \"9a42376a-140f-4b99-a2d2-e218ead49187\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") "
Sep 09 13:14:58.472585 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.472463    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-metrics-certs\") pod \"9a42376a-140f-4b99-a2d2-e218ead49187\" (UID: \"9a42376a-140f-4b99-a2d2-e218ead49187\") "
Sep 09 13:14:58.473049 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.473002    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a42376a-140f-4b99-a2d2-e218ead49187-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9a42376a-140f-4b99-a2d2-e218ead49187" (UID: "9a42376a-140f-4b99-a2d2-e218ead49187"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:58.474599 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.474574    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9a42376a-140f-4b99-a2d2-e218ead49187" (UID: "9a42376a-140f-4b99-a2d2-e218ead49187"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:14:58.475208 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.475186    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9a42376a-140f-4b99-a2d2-e218ead49187" (UID: "9a42376a-140f-4b99-a2d2-e218ead49187"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:14:58.475279 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.475217    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a42376a-140f-4b99-a2d2-e218ead49187-kube-api-access-gs4fq" (OuterVolumeSpecName: "kube-api-access-gs4fq") pod "9a42376a-140f-4b99-a2d2-e218ead49187" (UID: "9a42376a-140f-4b99-a2d2-e218ead49187"). InnerVolumeSpecName "kube-api-access-gs4fq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:14:58.574079 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.574053    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:58.574079 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.574080    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs4fq\" (UniqueName: \"kubernetes.io/projected/9a42376a-140f-4b99-a2d2-e218ead49187-kube-api-access-gs4fq\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:58.574079 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.574094    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a42376a-140f-4b99-a2d2-e218ead49187-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:58.574322 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.574107    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a42376a-140f-4b99-a2d2-e218ead49187-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:14:58.981090 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.980998    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a42376a-140f-4b99-a2d2-e218ead49187" containerID="81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565" exitCode=0
Sep 09 13:14:58.981090 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.981075    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"
Sep 09 13:14:58.981364 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.981042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b" event={"ID":"9a42376a-140f-4b99-a2d2-e218ead49187","Type":"ContainerDied","Data":"81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565"}
Sep 09 13:14:58.981364 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.981214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b" event={"ID":"9a42376a-140f-4b99-a2d2-e218ead49187","Type":"ContainerDied","Data":"e71a2479ea2e4defa2d49c94bb448ba2365a5ae37b8f895f826887c1c760da35"}
Sep 09 13:14:58.981364 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.981250    2577 scope.go:117] "RemoveContainer" containerID="81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565"
Sep 09 13:14:58.990414 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.990397    2577 scope.go:117] "RemoveContainer" containerID="81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565"
Sep 09 13:14:58.990698 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:14:58.990670    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565\": container with ID starting with 81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565 not found: ID does not exist" containerID="81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565"
Sep 09 13:14:58.990800 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.990701    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565"} err="failed to get container status \"81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565\": rpc error: code = NotFound desc = could not find container \"81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565\": container with ID starting with 81fd328123e7d6061b0f222a777af0de3b2487ecb4e4d4ebb9c5069bcb214565 not found: ID does not exist"
Sep 09 13:14:58.996746 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:58.996723    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"]
Sep 09 13:14:59.002277 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:14:59.002256    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-0b0a1a-8496b57765-9gx8b"]
Sep 09 13:15:00.134504 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.134472    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66"]
Sep 09 13:15:00.134908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.134829    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a42376a-140f-4b99-a2d2-e218ead49187" containerName="maas-api"
Sep 09 13:15:00.134908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.134842    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a42376a-140f-4b99-a2d2-e218ead49187" containerName="maas-api"
Sep 09 13:15:00.134908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.134907    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a42376a-140f-4b99-a2d2-e218ead49187" containerName="maas-api"
Sep 09 13:15:00.139078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.139060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66"
Sep 09 13:15:00.141371 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.141352    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:15:00.147114 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.146912    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66"]
Sep 09 13:15:00.151171 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.151152    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66"
Sep 09 13:15:00.420709 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.420686    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66"]
Sep 09 13:15:00.422451 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:15:00.422423    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2635f9_5c15_4a7f_a991_b73ba5fa0d2f.slice/crio-05df937d8e80a9d16c79934384cf43b09e7efc465c42b747c6b8d8919eafed28 WatchSource:0}: Error finding container 05df937d8e80a9d16c79934384cf43b09e7efc465c42b747c6b8d8919eafed28: Status 404 returned error can't find the container with id 05df937d8e80a9d16c79934384cf43b09e7efc465c42b747c6b8d8919eafed28
Sep 09 13:15:00.515015 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.514985    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a42376a-140f-4b99-a2d2-e218ead49187" path="/var/lib/kubelet/pods/9a42376a-140f-4b99-a2d2-e218ead49187/volumes"
Sep 09 13:15:00.994420 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.994388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66" event={"ID":"6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f","Type":"ContainerStarted","Data":"7da5982b1a74d397aa0d67226ea5c64c4f141b5f50f682e9d935c665482aa351"}
Sep 09 13:15:00.994575 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:00.994438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66" event={"ID":"6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f","Type":"ContainerStarted","Data":"05df937d8e80a9d16c79934384cf43b09e7efc465c42b747c6b8d8919eafed28"}
Sep 09 13:15:01.009883 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:01.009832    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66" podStartSLOduration=1.009809427 podStartE2EDuration="1.009809427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:15:01.007665172 +0000 UTC m=+1587.008331279" watchObservedRunningTime="2026-09-09 13:15:01.009809427 +0000 UTC m=+1587.010475551"
Sep 09 13:15:02.001545 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:02.001507    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f" containerID="7da5982b1a74d397aa0d67226ea5c64c4f141b5f50f682e9d935c665482aa351" exitCode=0
Sep 09 13:15:02.001944 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:02.001587    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66" event={"ID":"6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f","Type":"ContainerDied","Data":"7da5982b1a74d397aa0d67226ea5c64c4f141b5f50f682e9d935c665482aa351"}
Sep 09 13:15:03.136921 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:03.136898    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66"
Sep 09 13:15:04.012219 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:04.012187    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66"
Sep 09 13:15:04.012219 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:04.012206    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29815995-vlc66" event={"ID":"6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f","Type":"ContainerDied","Data":"05df937d8e80a9d16c79934384cf43b09e7efc465c42b747c6b8d8919eafed28"}
Sep 09 13:15:04.012469 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:04.012253    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05df937d8e80a9d16c79934384cf43b09e7efc465c42b747c6b8d8919eafed28"
Sep 09 13:15:04.916890 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:04.916852    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"]
Sep 09 13:15:04.917365 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:04.917124    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy" containerID="cri-o://0962b6041747493f0d929de38d231fa18b3110a1de57bee5d027de1d30517433" gracePeriod=30
Sep 09 13:15:07.986822 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:07.986786    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:15:10.050204 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.050167    2577 generic.go:356] "Generic (PLEG): container finished" podID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerID="0962b6041747493f0d929de38d231fa18b3110a1de57bee5d027de1d30517433" exitCode=0
Sep 09 13:15:10.050677 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.050257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" event={"ID":"988b91e4-6ada-44fb-a5be-e10475c24cbe","Type":"ContainerDied","Data":"0962b6041747493f0d929de38d231fa18b3110a1de57bee5d027de1d30517433"}
Sep 09 13:15:10.050677 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.050332    2577 scope.go:117] "RemoveContainer" containerID="be7e63b1be70bc63b1b9e29347675e636729e524de6eb242c662a89d27bf59fc"
Sep 09 13:15:10.184527 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.184454    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:15:10.280316 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280276    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zcfr\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-kube-api-access-6zcfr\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280316 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280323    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b91e4-6ada-44fb-a5be-e10475c24cbe-istiod-ca-cert\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280360    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-token\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280455    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-data\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280480    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-envoy\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280517    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-socket\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280547    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-certs\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280586 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-credential-socket\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280934 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280629    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-podinfo\") pod \"988b91e4-6ada-44fb-a5be-e10475c24cbe\" (UID: \"988b91e4-6ada-44fb-a5be-e10475c24cbe\") "
Sep 09 13:15:10.280934 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280841    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/988b91e4-6ada-44fb-a5be-e10475c24cbe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:15:10.281093 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.280944    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:10.281262 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.281129    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-data" (OuterVolumeSpecName: "istio-data") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:10.281262 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.281144    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:10.281262 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.281226    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:10.281485 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.281277    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988b91e4-6ada-44fb-a5be-e10475c24cbe-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:10.281485 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.281323    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:10.283010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.282981    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-token" (OuterVolumeSpecName: "istio-token") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:15:10.283184 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.283162    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:15:10.283268 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.283244    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:10.283355 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.283330    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-kube-api-access-6zcfr" (OuterVolumeSpecName: "kube-api-access-6zcfr") pod "988b91e4-6ada-44fb-a5be-e10475c24cbe" (UID: "988b91e4-6ada-44fb-a5be-e10475c24cbe"). InnerVolumeSpecName "kube-api-access-6zcfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:15:10.382597 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.382552    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:10.382597 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.382595    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:10.382908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.382609    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:10.382908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.382622    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988b91e4-6ada-44fb-a5be-e10475c24cbe-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:10.382908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.382654    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:10.382908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.382668    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zcfr\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-kube-api-access-6zcfr\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:10.382908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:10.382682    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988b91e4-6ada-44fb-a5be-e10475c24cbe-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:11.056872 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:11.056843    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"
Sep 09 13:15:11.057300 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:11.056839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb" event={"ID":"988b91e4-6ada-44fb-a5be-e10475c24cbe","Type":"ContainerDied","Data":"c4bb2e312ce9459ea4daa5d186a91abb9f17761c974ee136c06d8dc64cf168a7"}
Sep 09 13:15:11.057300 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:11.057005    2577 scope.go:117] "RemoveContainer" containerID="0962b6041747493f0d929de38d231fa18b3110a1de57bee5d027de1d30517433"
Sep 09 13:15:11.075101 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:11.075059    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"]
Sep 09 13:15:11.077363 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:11.077335    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-0b0a1a-openshift-default-64585cd4c6-k9njb"]
Sep 09 13:15:12.522873 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:12.522800    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" path="/var/lib/kubelet/pods/988b91e4-6ada-44fb-a5be-e10475c24cbe/volumes"
Sep 09 13:15:13.072545 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:13.072513    2577 generic.go:356] "Generic (PLEG): container finished" podID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerID="a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800" exitCode=0
Sep 09 13:15:13.072779 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:13.072589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" event={"ID":"76b8304a-7f81-40f8-aa68-de9b171547f8","Type":"ContainerDied","Data":"a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800"}
Sep 09 13:15:13.073063 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:13.073049    2577 scope.go:117] "RemoveContainer" containerID="a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800"
Sep 09 13:15:13.074228 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:13.074198    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:13.074283 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:13.074274    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:13.074324 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:13.074305    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:14.089350 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:14.089302    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" event={"ID":"76b8304a-7f81-40f8-aa68-de9b171547f8","Type":"ContainerStarted","Data":"a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03"}
Sep 09 13:15:15.036573 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:15.036538    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:15:15.036806 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:15.036784    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 09 13:15:15.036877 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:15.036852    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 09 13:15:15.687294 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:15.687252    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"]
Sep 09 13:15:15.688240 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:15.688173    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws" podUID="ba8fee37-6c76-4558-b530-4dae1aed42fe" containerName="maas-api" containerID="cri-o://f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8" gracePeriod=30
Sep 09 13:15:15.735528 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:15.735430    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"]
Sep 09 13:15:15.735684 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:15.735624    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq" podUID="6807b4d1-145f-436f-8fcf-e88ce3e372e6" containerName="payload-processing" containerID="cri-o://92b10bb8f7423001065496bf96d4f372846472dd9135c440f998e0f25a05dd17" gracePeriod=30
Sep 09 13:15:16.040085 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.039135    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 09 13:15:16.040085 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.039190    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 09 13:15:16.063758 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.063729    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:15:16.109056 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.109028    2577 generic.go:356] "Generic (PLEG): container finished" podID="6807b4d1-145f-436f-8fcf-e88ce3e372e6" containerID="92b10bb8f7423001065496bf96d4f372846472dd9135c440f998e0f25a05dd17" exitCode=0
Sep 09 13:15:16.109216 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.109163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq" event={"ID":"6807b4d1-145f-436f-8fcf-e88ce3e372e6","Type":"ContainerDied","Data":"92b10bb8f7423001065496bf96d4f372846472dd9135c440f998e0f25a05dd17"}
Sep 09 13:15:16.110780 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.110659    2577 generic.go:356] "Generic (PLEG): container finished" podID="ba8fee37-6c76-4558-b530-4dae1aed42fe" containerID="f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8" exitCode=0
Sep 09 13:15:16.110780 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.110735    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws" event={"ID":"ba8fee37-6c76-4558-b530-4dae1aed42fe","Type":"ContainerDied","Data":"f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8"}
Sep 09 13:15:16.110780 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.110766    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws" event={"ID":"ba8fee37-6c76-4558-b530-4dae1aed42fe","Type":"ContainerDied","Data":"a78cdaccfe38790c47e2b35fbbb48438eccda347d1dc67ecca42b793954a578e"}
Sep 09 13:15:16.111034 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.110784    2577 scope.go:117] "RemoveContainer" containerID="f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8"
Sep 09 13:15:16.111034 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.110952    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"
Sep 09 13:15:16.121318 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.121295    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:15:16.121747 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.121715    2577 scope.go:117] "RemoveContainer" containerID="f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8"
Sep 09 13:15:16.122025 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:15:16.122005    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8\": container with ID starting with f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8 not found: ID does not exist" containerID="f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8"
Sep 09 13:15:16.122104 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.122038    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8"} err="failed to get container status \"f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8\": rpc error: code = NotFound desc = could not find container \"f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8\": container with ID starting with f1995d476f35bae556fe092c1e4c1b8506b9fe8410dbf5fbecf88ac6c5940fc8 not found: ID does not exist"
Sep 09 13:15:16.241871 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.241781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls\") pod \"ba8fee37-6c76-4558-b530-4dae1aed42fe\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") "
Sep 09 13:15:16.241871 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.241825    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-metrics-certs\") pod \"ba8fee37-6c76-4558-b530-4dae1aed42fe\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") "
Sep 09 13:15:16.242103 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.241907    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvwn5\" (UniqueName: \"kubernetes.io/projected/ba8fee37-6c76-4558-b530-4dae1aed42fe-kube-api-access-bvwn5\") pod \"ba8fee37-6c76-4558-b530-4dae1aed42fe\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") "
Sep 09 13:15:16.242103 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.241949    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jfhb\" (UniqueName: \"kubernetes.io/projected/6807b4d1-145f-436f-8fcf-e88ce3e372e6-kube-api-access-8jfhb\") pod \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\" (UID: \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\") "
Sep 09 13:15:16.242103 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.241975    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba8fee37-6c76-4558-b530-4dae1aed42fe-openshift-service-ca\") pod \"ba8fee37-6c76-4558-b530-4dae1aed42fe\" (UID: \"ba8fee37-6c76-4558-b530-4dae1aed42fe\") "
Sep 09 13:15:16.242103 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.242025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6807b4d1-145f-436f-8fcf-e88ce3e372e6-plugins-config-volume\") pod \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\" (UID: \"6807b4d1-145f-436f-8fcf-e88ce3e372e6\") "
Sep 09 13:15:16.242594 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.242565    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba8fee37-6c76-4558-b530-4dae1aed42fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ba8fee37-6c76-4558-b530-4dae1aed42fe" (UID: "ba8fee37-6c76-4558-b530-4dae1aed42fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:15:16.242754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.242691    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6807b4d1-145f-436f-8fcf-e88ce3e372e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6807b4d1-145f-436f-8fcf-e88ce3e372e6" (UID: "6807b4d1-145f-436f-8fcf-e88ce3e372e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:15:16.244225 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.244197    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba8fee37-6c76-4558-b530-4dae1aed42fe-kube-api-access-bvwn5" (OuterVolumeSpecName: "kube-api-access-bvwn5") pod "ba8fee37-6c76-4558-b530-4dae1aed42fe" (UID: "ba8fee37-6c76-4558-b530-4dae1aed42fe"). InnerVolumeSpecName "kube-api-access-bvwn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:15:16.244553 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.244535    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ba8fee37-6c76-4558-b530-4dae1aed42fe" (UID: "ba8fee37-6c76-4558-b530-4dae1aed42fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:15:16.244720 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.244689    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ba8fee37-6c76-4558-b530-4dae1aed42fe" (UID: "ba8fee37-6c76-4558-b530-4dae1aed42fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:15:16.244797 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.244780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6807b4d1-145f-436f-8fcf-e88ce3e372e6-kube-api-access-8jfhb" (OuterVolumeSpecName: "kube-api-access-8jfhb") pod "6807b4d1-145f-436f-8fcf-e88ce3e372e6" (UID: "6807b4d1-145f-436f-8fcf-e88ce3e372e6"). InnerVolumeSpecName "kube-api-access-8jfhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:15:16.343543 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.343507    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:16.343543 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.343533    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba8fee37-6c76-4558-b530-4dae1aed42fe-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:16.343543 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.343542    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvwn5\" (UniqueName: \"kubernetes.io/projected/ba8fee37-6c76-4558-b530-4dae1aed42fe-kube-api-access-bvwn5\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:16.343543 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.343551    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jfhb\" (UniqueName: \"kubernetes.io/projected/6807b4d1-145f-436f-8fcf-e88ce3e372e6-kube-api-access-8jfhb\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:16.343824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.343560    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba8fee37-6c76-4558-b530-4dae1aed42fe-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:16.343824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.343568    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6807b4d1-145f-436f-8fcf-e88ce3e372e6-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:15:16.431577 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.431547    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"]
Sep 09 13:15:16.435419 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.435395    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-941953-64fc5cc7db-6lfws"]
Sep 09 13:15:16.516369 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:16.516293    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba8fee37-6c76-4558-b530-4dae1aed42fe" path="/var/lib/kubelet/pods/ba8fee37-6c76-4558-b530-4dae1aed42fe/volumes"
Sep 09 13:15:17.037794 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:17.037758    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 09 13:15:17.038154 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:17.037813    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 09 13:15:17.119059 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:17.119025    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq" event={"ID":"6807b4d1-145f-436f-8fcf-e88ce3e372e6","Type":"ContainerDied","Data":"11bd64d90c2f3c39528ff29557399c58b37fb11ec41af31e452e182f5e1604ee"}
Sep 09 13:15:17.119212 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:17.119077    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"
Sep 09 13:15:17.119212 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:17.119104    2577 scope.go:117] "RemoveContainer" containerID="92b10bb8f7423001065496bf96d4f372846472dd9135c440f998e0f25a05dd17"
Sep 09 13:15:17.149966 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:17.149932    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"]
Sep 09 13:15:17.156444 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:17.156422    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-941953-7fc46d7ccc-vdqrq"]
Sep 09 13:15:18.037465 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:18.037422    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 09 13:15:18.037711 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:18.037489    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 09 13:15:18.514854 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:18.514825    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6807b4d1-145f-436f-8fcf-e88ce3e372e6" path="/var/lib/kubelet/pods/6807b4d1-145f-436f-8fcf-e88ce3e372e6/volumes"
Sep 09 13:15:19.037169 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.037133    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Sep 09 13:15:19.037345 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.037193    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Sep 09 13:15:19.827070 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827032    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"]
Sep 09 13:15:19.827544 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827524    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f" containerName="cleanup"
Sep 09 13:15:19.827624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827548    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f" containerName="cleanup"
Sep 09 13:15:19.827624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827566    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy"
Sep 09 13:15:19.827624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827575    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy"
Sep 09 13:15:19.827624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827586    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6807b4d1-145f-436f-8fcf-e88ce3e372e6" containerName="payload-processing"
Sep 09 13:15:19.827624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827594    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6807b4d1-145f-436f-8fcf-e88ce3e372e6" containerName="payload-processing"
Sep 09 13:15:19.827624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827606    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy"
Sep 09 13:15:19.827624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827614    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy"
Sep 09 13:15:19.827984 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827629    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba8fee37-6c76-4558-b530-4dae1aed42fe" containerName="maas-api"
Sep 09 13:15:19.827984 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827669    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba8fee37-6c76-4558-b530-4dae1aed42fe" containerName="maas-api"
Sep 09 13:15:19.827984 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827799    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6807b4d1-145f-436f-8fcf-e88ce3e372e6" containerName="payload-processing"
Sep 09 13:15:19.827984 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827813    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy"
Sep 09 13:15:19.827984 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827824    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba8fee37-6c76-4558-b530-4dae1aed42fe" containerName="maas-api"
Sep 09 13:15:19.827984 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827835    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="988b91e4-6ada-44fb-a5be-e10475c24cbe" containerName="istio-proxy"
Sep 09 13:15:19.827984 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.827846    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f" containerName="cleanup"
Sep 09 13:15:19.831356 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.831334    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.833893 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.833871    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-ae6936-openshift-default-dockercfg-rzkwt\""
Sep 09 13:15:19.841514 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.841491    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"]
Sep 09 13:15:19.873499 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-data\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.873669 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-socket\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.873669 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc57d\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-kube-api-access-jc57d\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.873792 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-credential-socket\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.873792 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-envoy\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.873792 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873782    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-certs\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.873915 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873806    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-podinfo\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.873915 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67d0e76-4751-4e18-bb72-976c7e4c107b-istiod-ca-cert\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.873915 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.873865    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-token\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975006 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.974969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-token\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975223 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-data\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975223 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-socket\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975223 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc57d\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-kube-api-access-jc57d\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975502 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-data\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975565 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975534    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-socket\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975565 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-credential-socket\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975699 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975583    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-envoy\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975699 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-certs\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975699 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-podinfo\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975859 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67d0e76-4751-4e18-bb72-976c7e4c107b-istiod-ca-cert\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975859 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975813    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-credential-socket\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.975975 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.975950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-certs\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.976315 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.976290    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67d0e76-4751-4e18-bb72-976c7e4c107b-istiod-ca-cert\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.977928 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.977900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-envoy\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.978212 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.978197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-podinfo\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.987769 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.987740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc57d\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-kube-api-access-jc57d\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:19.987854 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:19.987803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-token\") pod \"e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") " pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:20.041079 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:20.041061    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:15:20.041352 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:20.041335    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:15:20.041959 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:20.041944    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:15:20.144390 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:20.144362    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:20.452728 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:20.448978    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"]
Sep 09 13:15:20.455555 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:15:20.455525    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode67d0e76_4751_4e18_bb72_976c7e4c107b.slice/crio-49b69fc0a406ad4fd9cf67044b77eb1acf2eca381bbb35d4e75d108624d5126c WatchSource:0}: Error finding container 49b69fc0a406ad4fd9cf67044b77eb1acf2eca381bbb35d4e75d108624d5126c: Status 404 returned error can't find the container with id 49b69fc0a406ad4fd9cf67044b77eb1acf2eca381bbb35d4e75d108624d5126c
Sep 09 13:15:20.457937 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:20.457908    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:20.458005 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:20.457968    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:20.458005 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:20.457995    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:21.143060 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:21.143024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" event={"ID":"e67d0e76-4751-4e18-bb72-976c7e4c107b","Type":"ContainerStarted","Data":"cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d"}
Sep 09 13:15:21.143527 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:21.143076    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" event={"ID":"e67d0e76-4751-4e18-bb72-976c7e4c107b","Type":"ContainerStarted","Data":"49b69fc0a406ad4fd9cf67044b77eb1acf2eca381bbb35d4e75d108624d5126c"}
Sep 09 13:15:21.145497 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:21.145481    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:21.149956 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:21.149934    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:21.163800 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:21.163762    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" podStartSLOduration=2.163751015 podStartE2EDuration="2.163751015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:15:21.162296669 +0000 UTC m=+1607.162962768" watchObservedRunningTime="2026-09-09 13:15:21.163751015 +0000 UTC m=+1607.164417114"
Sep 09 13:15:22.148511 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:22.148479    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:22.149368 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:22.149346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:24.361419 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.361389    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"]
Sep 09 13:15:24.363784 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.363765    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.366029 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.366010    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-ae6936-serving-cert\""
Sep 09 13:15:24.375910 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.375888    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"]
Sep 09 13:15:24.416374 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.416351    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-metrics-certs\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.416485 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.416381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa219116-aacf-45e4-9909-79a6c8fa8816-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.416485 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.416430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl49q\" (UniqueName: \"kubernetes.io/projected/aa219116-aacf-45e4-9909-79a6c8fa8816-kube-api-access-tl49q\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.416485 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.416458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.517568 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.517540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-metrics-certs\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.517568 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.517572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa219116-aacf-45e4-9909-79a6c8fa8816-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.517813 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.517607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl49q\" (UniqueName: \"kubernetes.io/projected/aa219116-aacf-45e4-9909-79a6c8fa8816-kube-api-access-tl49q\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.517813 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.517628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.518198 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.518179    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa219116-aacf-45e4-9909-79a6c8fa8816-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.520384 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.520362    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-metrics-certs\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.520704 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.520687    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-maas-api-tls\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.525677 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.525653    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl49q\" (UniqueName: \"kubernetes.io/projected/aa219116-aacf-45e4-9909-79a6c8fa8816-kube-api-access-tl49q\") pod \"maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.676526 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.676498    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:24.954520 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.954493    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"]
Sep 09 13:15:24.956201 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:15:24.956171    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa219116_aacf_45e4_9909_79a6c8fa8816.slice/crio-5823cfe011bf6f88181d5e21eded8aeea08e269974a80cc0842464480b2451d8 WatchSource:0}: Error finding container 5823cfe011bf6f88181d5e21eded8aeea08e269974a80cc0842464480b2451d8: Status 404 returned error can't find the container with id 5823cfe011bf6f88181d5e21eded8aeea08e269974a80cc0842464480b2451d8
Sep 09 13:15:24.957915 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:24.957894    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:15:25.166411 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:25.166379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64" event={"ID":"aa219116-aacf-45e4-9909-79a6c8fa8816","Type":"ContainerStarted","Data":"5823cfe011bf6f88181d5e21eded8aeea08e269974a80cc0842464480b2451d8"}
Sep 09 13:15:26.172898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:26.172858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64" event={"ID":"aa219116-aacf-45e4-9909-79a6c8fa8816","Type":"ContainerStarted","Data":"2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3"}
Sep 09 13:15:26.173290 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:26.172917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:26.189909 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:26.189861    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64" podStartSLOduration=1.535854184 podStartE2EDuration="2.189845949s" totalImagesPullingTime="653.991765ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:15:26.188096967 +0000 UTC m=+1612.188763067" watchObservedRunningTime="2026-09-09 13:15:26.189845949 +0000 UTC m=+1612.190512049"
Sep 09 13:15:32.185285 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:32.185251    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:15:42.255197 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:42.255159    2577 generic.go:356] "Generic (PLEG): container finished" podID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerID="cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d" exitCode=0
Sep 09 13:15:42.255617 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:42.255235    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" event={"ID":"e67d0e76-4751-4e18-bb72-976c7e4c107b","Type":"ContainerDied","Data":"cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d"}
Sep 09 13:15:42.255729 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:42.255712    2577 scope.go:117] "RemoveContainer" containerID="cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d"
Sep 09 13:15:42.256835 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:42.256799    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:42.256904 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:42.256870    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:42.256941 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:42.256902    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:15:43.261284 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:43.261251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" event={"ID":"e67d0e76-4751-4e18-bb72-976c7e4c107b","Type":"ContainerStarted","Data":"29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb"}
Sep 09 13:15:44.145538 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:44.145503    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:44.145829 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:44.145767    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:15:44.145891 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:44.145820    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 09 13:15:45.144999 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:45.144961    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:15:45.145404 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:45.145018    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 09 13:15:46.144901 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:46.144864    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:15:46.145089 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:46.144920    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 09 13:15:47.145787 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:47.145751    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:15:47.146173 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:47.145809    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 09 13:15:48.145251 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:48.145215    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:15:48.145435 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:48.145271    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 09 13:15:49.149199 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:49.149166    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:49.149683 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:49.149458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:15:49.150212 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:15:49.150193    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:16:34.258056 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:34.258018    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q_a0fa2864-cd6d-45d9-9829-89aa102d4e8e/payload-processing/0.log"
Sep 09 13:16:35.066665 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.066600    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"]
Sep 09 13:16:35.069137 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.069112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.071377 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.071353    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-964720-openshift-default-dockercfg-7wtx8\""
Sep 09 13:16:35.081398 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.081373    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"]
Sep 09 13:16:35.180740 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.180708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-socket\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.181024 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.181001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-certs\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.181193 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.181162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-credential-socket\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.181335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.181212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-podinfo\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.181335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.181268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbfc0556-116e-4acf-94ae-95eeea61eff5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.181335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.181299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-data\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.181523 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.181365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-token\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.181523 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.181430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lft6\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-kube-api-access-6lft6\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.181523 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.181485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-envoy\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.282408 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lft6\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-kube-api-access-6lft6\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.282408 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282422    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-envoy\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-socket\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-certs\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-credential-socket\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-podinfo\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbfc0556-116e-4acf-94ae-95eeea61eff5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-data\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.282618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-token\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283393 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.283008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-credential-socket\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283393 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.283039    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-socket\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283393 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.283141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-data\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283522 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.283421    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-certs\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.283600 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.283579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbfc0556-116e-4acf-94ae-95eeea61eff5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.285064 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.285042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-envoy\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.285256 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.285239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-podinfo\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.290325 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.290294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-token\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.290431 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.290404    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lft6\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-kube-api-access-6lft6\") pod \"e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") " pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.380413 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.380381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:35.655270 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.655238    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"]
Sep 09 13:16:35.656594 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:16:35.656564    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbfc0556_116e_4acf_94ae_95eeea61eff5.slice/crio-5edf8b3a54d8991919fbde2812976d916aac985e9b26e641bd73559761587401 WatchSource:0}: Error finding container 5edf8b3a54d8991919fbde2812976d916aac985e9b26e641bd73559761587401: Status 404 returned error can't find the container with id 5edf8b3a54d8991919fbde2812976d916aac985e9b26e641bd73559761587401
Sep 09 13:16:35.658840 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.658797    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:16:35.658967 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.658866    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:16:35.658967 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:35.658894    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:16:36.545257 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:36.545222    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" event={"ID":"dbfc0556-116e-4acf-94ae-95eeea61eff5","Type":"ContainerStarted","Data":"ac862612120f07b03e36a37454dd93f8039ba5f16ba5d03d314621f400155fae"}
Sep 09 13:16:36.545783 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:36.545283    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" event={"ID":"dbfc0556-116e-4acf-94ae-95eeea61eff5","Type":"ContainerStarted","Data":"5edf8b3a54d8991919fbde2812976d916aac985e9b26e641bd73559761587401"}
Sep 09 13:16:36.563748 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:36.563693    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" podStartSLOduration=1.563672046 podStartE2EDuration="1.563672046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:16:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:16:36.562248044 +0000 UTC m=+1682.562914145" watchObservedRunningTime="2026-09-09 13:16:36.563672046 +0000 UTC m=+1682.564338149"
Sep 09 13:16:37.381443 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:37.381400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:37.387292 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:37.387266    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:37.549789 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:37.549756    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:37.550990 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:37.550967    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:57.648363 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:57.648321    2577 generic.go:356] "Generic (PLEG): container finished" podID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerID="ac862612120f07b03e36a37454dd93f8039ba5f16ba5d03d314621f400155fae" exitCode=0
Sep 09 13:16:57.648807 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:57.648396    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" event={"ID":"dbfc0556-116e-4acf-94ae-95eeea61eff5","Type":"ContainerDied","Data":"ac862612120f07b03e36a37454dd93f8039ba5f16ba5d03d314621f400155fae"}
Sep 09 13:16:57.648908 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:57.648891    2577 scope.go:117] "RemoveContainer" containerID="ac862612120f07b03e36a37454dd93f8039ba5f16ba5d03d314621f400155fae"
Sep 09 13:16:57.650081 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:57.650049    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:16:57.650196 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:57.650133    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:16:57.650196 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:57.650183    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 09 13:16:58.655824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:58.655792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" event={"ID":"dbfc0556-116e-4acf-94ae-95eeea61eff5","Type":"ContainerStarted","Data":"31344e3a23c08c48440786d2cbeb02dc6874c0251b11e21421c203cd901484e4"}
Sep 09 13:16:59.381171 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:59.381125    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:16:59.381463 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:59.381438    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:16:59.381708 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:16:59.381675    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:17:00.381259 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:00.381223    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:17:00.381877 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:00.381846    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:17:01.381135 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:01.381095    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:17:01.381486 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:01.381447    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:17:02.381797 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:02.381746    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:17:02.382366 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:02.381825    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:17:03.381167 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:03.381126    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:17:03.381532 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:03.381495    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:17:04.385845 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:04.385816    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:17:04.386271 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:04.386076    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:17:04.386815 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:04.386800    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:17:16.640650 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:16.640592    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"]
Sep 09 13:17:16.641207 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:16.640906    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy" containerID="cri-o://31344e3a23c08c48440786d2cbeb02dc6874c0251b11e21421c203cd901484e4" gracePeriod=30
Sep 09 13:17:19.389296 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:19.389250    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 09 13:17:19.389763 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:19.389313    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 13:17:21.783200 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.783163    2577 generic.go:356] "Generic (PLEG): container finished" podID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerID="31344e3a23c08c48440786d2cbeb02dc6874c0251b11e21421c203cd901484e4" exitCode=0
Sep 09 13:17:21.783727 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.783230    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" event={"ID":"dbfc0556-116e-4acf-94ae-95eeea61eff5","Type":"ContainerDied","Data":"31344e3a23c08c48440786d2cbeb02dc6874c0251b11e21421c203cd901484e4"}
Sep 09 13:17:21.783727 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.783310    2577 scope.go:117] "RemoveContainer" containerID="ac862612120f07b03e36a37454dd93f8039ba5f16ba5d03d314621f400155fae"
Sep 09 13:17:21.899883 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.899854    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:17:21.938522 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938413    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-certs\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.938522 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938454    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-envoy\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.938522 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938501    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lft6\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-kube-api-access-6lft6\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.938824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938552    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbfc0556-116e-4acf-94ae-95eeea61eff5-istiod-ca-cert\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.938824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938693    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-podinfo\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.938824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938728    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:21.938824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-data\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.938824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938812    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-credential-socket\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.939060 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938852    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-socket\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.939060 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-token\") pod \"dbfc0556-116e-4acf-94ae-95eeea61eff5\" (UID: \"dbfc0556-116e-4acf-94ae-95eeea61eff5\") "
Sep 09 13:17:21.939060 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.938952    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbfc0556-116e-4acf-94ae-95eeea61eff5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:17:21.939208 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.939141    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:21.939208 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.939197    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:21.939411 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.939388    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-data" (OuterVolumeSpecName: "istio-data") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:21.939681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.939499    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbfc0556-116e-4acf-94ae-95eeea61eff5-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:21.939807 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.939696    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:21.939807 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.939715    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:21.939807 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.939733    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:21.941197 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.941160    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:21.941502 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.941474    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-token" (OuterVolumeSpecName: "istio-token") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:17:21.941603 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.941537    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-kube-api-access-6lft6" (OuterVolumeSpecName: "kube-api-access-6lft6") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "kube-api-access-6lft6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:17:21.941603 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:21.941543    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dbfc0556-116e-4acf-94ae-95eeea61eff5" (UID: "dbfc0556-116e-4acf-94ae-95eeea61eff5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:17:22.040776 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.040737    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:22.040776 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.040768    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:22.040776 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.040776    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:22.040776 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.040785    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lft6\" (UniqueName: \"kubernetes.io/projected/dbfc0556-116e-4acf-94ae-95eeea61eff5-kube-api-access-6lft6\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:22.041067 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.040794    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbfc0556-116e-4acf-94ae-95eeea61eff5-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:22.790385 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.790299    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"
Sep 09 13:17:22.790385 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.790330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4" event={"ID":"dbfc0556-116e-4acf-94ae-95eeea61eff5","Type":"ContainerDied","Data":"5edf8b3a54d8991919fbde2812976d916aac985e9b26e641bd73559761587401"}
Sep 09 13:17:22.790385 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.790387    2577 scope.go:117] "RemoveContainer" containerID="31344e3a23c08c48440786d2cbeb02dc6874c0251b11e21421c203cd901484e4"
Sep 09 13:17:22.807564 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.807532    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"]
Sep 09 13:17:22.813162 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:22.813132    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-964720-openshift-default-845cb694dd-xxpw4"]
Sep 09 13:17:24.517787 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:24.517750    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" path="/var/lib/kubelet/pods/dbfc0556-116e-4acf-94ae-95eeea61eff5/volumes"
Sep 09 13:17:36.046595 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.046545    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"]
Sep 09 13:17:36.047066 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.046833    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q" podUID="a0fa2864-cd6d-45d9-9829-89aa102d4e8e" containerName="payload-processing" containerID="cri-o://f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c" gracePeriod=30
Sep 09 13:17:36.344406 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.344378    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:17:36.488459 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.488432    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-plugins-config-volume\") pod \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\" (UID: \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\") "
Sep 09 13:17:36.488676 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.488496    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqdh5\" (UniqueName: \"kubernetes.io/projected/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-kube-api-access-xqdh5\") pod \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\" (UID: \"a0fa2864-cd6d-45d9-9829-89aa102d4e8e\") "
Sep 09 13:17:36.488897 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.488873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0fa2864-cd6d-45d9-9829-89aa102d4e8e" (UID: "a0fa2864-cd6d-45d9-9829-89aa102d4e8e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:17:36.490805 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.490784    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-kube-api-access-xqdh5" (OuterVolumeSpecName: "kube-api-access-xqdh5") pod "a0fa2864-cd6d-45d9-9829-89aa102d4e8e" (UID: "a0fa2864-cd6d-45d9-9829-89aa102d4e8e"). InnerVolumeSpecName "kube-api-access-xqdh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:17:36.590135 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.590024    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:36.590135 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.590054    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqdh5\" (UniqueName: \"kubernetes.io/projected/a0fa2864-cd6d-45d9-9829-89aa102d4e8e-kube-api-access-xqdh5\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:17:36.863855 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.863735    2577 generic.go:356] "Generic (PLEG): container finished" podID="a0fa2864-cd6d-45d9-9829-89aa102d4e8e" containerID="f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c" exitCode=0
Sep 09 13:17:36.863855 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.863794    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"
Sep 09 13:17:36.863855 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.863827    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q" event={"ID":"a0fa2864-cd6d-45d9-9829-89aa102d4e8e","Type":"ContainerDied","Data":"f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c"}
Sep 09 13:17:36.864283 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.863892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q" event={"ID":"a0fa2864-cd6d-45d9-9829-89aa102d4e8e","Type":"ContainerDied","Data":"9b994cbba02b6a984ef3de29f26f79a84761e1f0b01e7a107cabcd13d9128cfc"}
Sep 09 13:17:36.864283 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.863919    2577 scope.go:117] "RemoveContainer" containerID="f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c"
Sep 09 13:17:36.873608 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.873584    2577 scope.go:117] "RemoveContainer" containerID="f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c"
Sep 09 13:17:36.874012 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:17:36.873987    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c\": container with ID starting with f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c not found: ID does not exist" containerID="f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c"
Sep 09 13:17:36.874094 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.874025    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c"} err="failed to get container status \"f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c\": rpc error: code = NotFound desc = could not find container \"f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c\": container with ID starting with f6a3d8d0da968a01d8a5a8ee89f8add68b3970ee09f651e8f340bc1504a4687c not found: ID does not exist"
Sep 09 13:17:36.880284 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.880257    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"]
Sep 09 13:17:36.882195 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:36.882172    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-458b0f-869c5c8d67-n2g8q"]
Sep 09 13:17:38.515928 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:17:38.515895    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0fa2864-cd6d-45d9-9829-89aa102d4e8e" path="/var/lib/kubelet/pods/a0fa2864-cd6d-45d9-9829-89aa102d4e8e/volumes"
Sep 09 13:18:24.253826 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:24.253787    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"]
Sep 09 13:18:24.254404 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:24.254091    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy" containerID="cri-o://a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03" gracePeriod=30
Sep 09 13:18:29.497702 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.497675    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:18:29.654698 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.654668    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-credential-socket\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.654877 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.654742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-socket\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.654877 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.654769    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-podinfo\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.654877 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.654812    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b8304a-7f81-40f8-aa68-de9b171547f8-istiod-ca-cert\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.654877 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.654852    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-data\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.655100 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.654893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-certs\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.655100 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.654967    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:29.655100 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.654985    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-token\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.655100 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655025    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2t4b\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-kube-api-access-s2t4b\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.655100 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655065    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-envoy\") pod \"76b8304a-7f81-40f8-aa68-de9b171547f8\" (UID: \"76b8304a-7f81-40f8-aa68-de9b171547f8\") "
Sep 09 13:18:29.655360 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655113    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:29.655360 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655188    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:29.655360 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655202    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:29.655360 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655246    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76b8304a-7f81-40f8-aa68-de9b171547f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:18:29.655754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655439    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b8304a-7f81-40f8-aa68-de9b171547f8-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.655754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655459    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.655754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655473    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.655754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655485    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.655754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.655499    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.657279 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.657249    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:18:29.657279 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.657257    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:29.657460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.657296    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:18:29.657460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.657411    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-kube-api-access-s2t4b" (OuterVolumeSpecName: "kube-api-access-s2t4b") pod "76b8304a-7f81-40f8-aa68-de9b171547f8" (UID: "76b8304a-7f81-40f8-aa68-de9b171547f8"). InnerVolumeSpecName "kube-api-access-s2t4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:18:29.756070 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.756015    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.756070 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.756064    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.756070 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.756077    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.756326 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:29.756089    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2t4b\" (UniqueName: \"kubernetes.io/projected/76b8304a-7f81-40f8-aa68-de9b171547f8-kube-api-access-s2t4b\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:30.122502 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.122401    2577 generic.go:356] "Generic (PLEG): container finished" podID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerID="a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03" exitCode=0
Sep 09 13:18:30.122502 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.122457    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"
Sep 09 13:18:30.122502 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.122466    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" event={"ID":"76b8304a-7f81-40f8-aa68-de9b171547f8","Type":"ContainerDied","Data":"a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03"}
Sep 09 13:18:30.122810 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.122510    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9" event={"ID":"76b8304a-7f81-40f8-aa68-de9b171547f8","Type":"ContainerDied","Data":"c3b94ff9723b094798bc64843c6d57ab47844c870038e51455e6e71ffc7e568f"}
Sep 09 13:18:30.122810 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.122532    2577 scope.go:117] "RemoveContainer" containerID="a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03"
Sep 09 13:18:30.131920 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.131904    2577 scope.go:117] "RemoveContainer" containerID="a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800"
Sep 09 13:18:30.139513 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.139497    2577 scope.go:117] "RemoveContainer" containerID="a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03"
Sep 09 13:18:30.139773 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:18:30.139758    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03\": container with ID starting with a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03 not found: ID does not exist" containerID="a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03"
Sep 09 13:18:30.139823 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.139780    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03"} err="failed to get container status \"a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03\": rpc error: code = NotFound desc = could not find container \"a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03\": container with ID starting with a48a83cd32af23ef6948d02ccbd899925cc871556ca42febff52fea79cd76f03 not found: ID does not exist"
Sep 09 13:18:30.139823 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.139795    2577 scope.go:117] "RemoveContainer" containerID="a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800"
Sep 09 13:18:30.140019 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:18:30.140001    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800\": container with ID starting with a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800 not found: ID does not exist" containerID="a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800"
Sep 09 13:18:30.140075 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.140028    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800"} err="failed to get container status \"a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800\": rpc error: code = NotFound desc = could not find container \"a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800\": container with ID starting with a90ad56f061160663c5db7a840eef45b75c1212ff74960b9229ca0823aa74800 not found: ID does not exist"
Sep 09 13:18:30.143461 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.143428    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"]
Sep 09 13:18:30.146544 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.146520    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-458b0f-openshift-default-5c88f867f6-d67c9"]
Sep 09 13:18:30.515610 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:30.515580    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" path="/var/lib/kubelet/pods/76b8304a-7f81-40f8-aa68-de9b171547f8/volumes"
Sep 09 13:18:35.093547 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.093462    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"]
Sep 09 13:18:35.094309 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.094249    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64" podUID="aa219116-aacf-45e4-9909-79a6c8fa8816" containerName="maas-api" containerID="cri-o://2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3" gracePeriod=30
Sep 09 13:18:35.337414 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.337384    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:18:35.500414 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.500348    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa219116-aacf-45e4-9909-79a6c8fa8816-openshift-service-ca\") pod \"aa219116-aacf-45e4-9909-79a6c8fa8816\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") "
Sep 09 13:18:35.500414 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.500411    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl49q\" (UniqueName: \"kubernetes.io/projected/aa219116-aacf-45e4-9909-79a6c8fa8816-kube-api-access-tl49q\") pod \"aa219116-aacf-45e4-9909-79a6c8fa8816\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") "
Sep 09 13:18:35.500745 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.500446    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-metrics-certs\") pod \"aa219116-aacf-45e4-9909-79a6c8fa8816\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") "
Sep 09 13:18:35.500745 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.500499    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-maas-api-tls\") pod \"aa219116-aacf-45e4-9909-79a6c8fa8816\" (UID: \"aa219116-aacf-45e4-9909-79a6c8fa8816\") "
Sep 09 13:18:35.500916 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.500811    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa219116-aacf-45e4-9909-79a6c8fa8816-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa219116-aacf-45e4-9909-79a6c8fa8816" (UID: "aa219116-aacf-45e4-9909-79a6c8fa8816"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:18:35.500996 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.500924    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa219116-aacf-45e4-9909-79a6c8fa8816-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:35.503692 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.503661    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa219116-aacf-45e4-9909-79a6c8fa8816" (UID: "aa219116-aacf-45e4-9909-79a6c8fa8816"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:18:35.503692 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.503679    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa219116-aacf-45e4-9909-79a6c8fa8816-kube-api-access-tl49q" (OuterVolumeSpecName: "kube-api-access-tl49q") pod "aa219116-aacf-45e4-9909-79a6c8fa8816" (UID: "aa219116-aacf-45e4-9909-79a6c8fa8816"). InnerVolumeSpecName "kube-api-access-tl49q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:18:35.504084 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.504057    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aa219116-aacf-45e4-9909-79a6c8fa8816" (UID: "aa219116-aacf-45e4-9909-79a6c8fa8816"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:18:35.602078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.602041    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl49q\" (UniqueName: \"kubernetes.io/projected/aa219116-aacf-45e4-9909-79a6c8fa8816-kube-api-access-tl49q\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:35.602078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.602071    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:35.602078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:35.602080    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa219116-aacf-45e4-9909-79a6c8fa8816-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:36.156185 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.156145    2577 generic.go:356] "Generic (PLEG): container finished" podID="aa219116-aacf-45e4-9909-79a6c8fa8816" containerID="2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3" exitCode=0
Sep 09 13:18:36.156185 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.156186    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"
Sep 09 13:18:36.156721 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.156237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64" event={"ID":"aa219116-aacf-45e4-9909-79a6c8fa8816","Type":"ContainerDied","Data":"2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3"}
Sep 09 13:18:36.156721 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.156295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64" event={"ID":"aa219116-aacf-45e4-9909-79a6c8fa8816","Type":"ContainerDied","Data":"5823cfe011bf6f88181d5e21eded8aeea08e269974a80cc0842464480b2451d8"}
Sep 09 13:18:36.156721 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.156329    2577 scope.go:117] "RemoveContainer" containerID="2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3"
Sep 09 13:18:36.165531 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.165512    2577 scope.go:117] "RemoveContainer" containerID="2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3"
Sep 09 13:18:36.165842 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:18:36.165821    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3\": container with ID starting with 2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3 not found: ID does not exist" containerID="2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3"
Sep 09 13:18:36.165925 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.165848    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3"} err="failed to get container status \"2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3\": rpc error: code = NotFound desc = could not find container \"2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3\": container with ID starting with 2e77b07ea4421f5efa092ba4956e414decbc69a74028d5c3b8744abd284b36a3 not found: ID does not exist"
Sep 09 13:18:36.176398 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.176365    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"]
Sep 09 13:18:36.179285 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.179266    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-ae6936-665dd96d9c-k8t64"]
Sep 09 13:18:36.515579 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:36.515505    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa219116-aacf-45e4-9909-79a6c8fa8816" path="/var/lib/kubelet/pods/aa219116-aacf-45e4-9909-79a6c8fa8816/volumes"
Sep 09 13:18:41.778694 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:41.778657    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"]
Sep 09 13:18:41.779269 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:41.778937    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy" containerID="cri-o://29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb" gracePeriod=30
Sep 09 13:18:47.019842 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.019818    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:18:47.203598 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203565    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc57d\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-kube-api-access-jc57d\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.203755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203664    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-socket\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.203755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203709    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-envoy\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.203755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203737    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-data\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.203755 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67d0e76-4751-4e18-bb72-976c7e4c107b-istiod-ca-cert\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.203981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203773    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-podinfo\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.203981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203792    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-certs\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.203981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203825    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-credential-socket\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.203981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.203853    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-token\") pod \"e67d0e76-4751-4e18-bb72-976c7e4c107b\" (UID: \"e67d0e76-4751-4e18-bb72-976c7e4c107b\") "
Sep 09 13:18:47.204204 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.204023    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:47.204204 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.204058    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-data" (OuterVolumeSpecName: "istio-data") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:47.204318 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.204237    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:47.204318 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.204245    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:47.204318 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.204253    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:47.204481 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.204434    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:47.204593 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.204574    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e67d0e76-4751-4e18-bb72-976c7e4c107b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:18:47.206534 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.206510    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-kube-api-access-jc57d" (OuterVolumeSpecName: "kube-api-access-jc57d") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "kube-api-access-jc57d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:18:47.206655 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.206548    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:18:47.206761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.206668    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-token" (OuterVolumeSpecName: "istio-token") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:18:47.206761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.206725    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e67d0e76-4751-4e18-bb72-976c7e4c107b" (UID: "e67d0e76-4751-4e18-bb72-976c7e4c107b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:18:47.209688 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.209663    2577 generic.go:356] "Generic (PLEG): container finished" podID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerID="29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb" exitCode=0
Sep 09 13:18:47.209788 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.209701    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"
Sep 09 13:18:47.209788 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.209729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" event={"ID":"e67d0e76-4751-4e18-bb72-976c7e4c107b","Type":"ContainerDied","Data":"29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb"}
Sep 09 13:18:47.209913 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.209791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv" event={"ID":"e67d0e76-4751-4e18-bb72-976c7e4c107b","Type":"ContainerDied","Data":"49b69fc0a406ad4fd9cf67044b77eb1acf2eca381bbb35d4e75d108624d5126c"}
Sep 09 13:18:47.209913 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.209823    2577 scope.go:117] "RemoveContainer" containerID="29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb"
Sep 09 13:18:47.228355 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.228338    2577 scope.go:117] "RemoveContainer" containerID="cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d"
Sep 09 13:18:47.236703 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.236685    2577 scope.go:117] "RemoveContainer" containerID="29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb"
Sep 09 13:18:47.236955 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:18:47.236936    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb\": container with ID starting with 29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb not found: ID does not exist" containerID="29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb"
Sep 09 13:18:47.237014 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.236963    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb"} err="failed to get container status \"29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb\": rpc error: code = NotFound desc = could not find container \"29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb\": container with ID starting with 29bcebaf7e4b539f36a2c3ba297f5b3fcc99e5c07ac280db782300e95a4b82fb not found: ID does not exist"
Sep 09 13:18:47.237014 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.236980    2577 scope.go:117] "RemoveContainer" containerID="cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d"
Sep 09 13:18:47.237226 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:18:47.237211    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d\": container with ID starting with cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d not found: ID does not exist" containerID="cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d"
Sep 09 13:18:47.237266 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.237229    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d"} err="failed to get container status \"cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d\": rpc error: code = NotFound desc = could not find container \"cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d\": container with ID starting with cf86b8e763b738ba9b45ce337d15fd4aef6505fdecceb1c002a0e19812c2558d not found: ID does not exist"
Sep 09 13:18:47.238067 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.238049    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"]
Sep 09 13:18:47.243247 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.243228    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-ae6936-openshift-default-58f4c94c68-2jpkv"]
Sep 09 13:18:47.304844 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.304823    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:47.304844 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.304843    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67d0e76-4751-4e18-bb72-976c7e4c107b-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:47.304960 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.304851    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:47.304960 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.304859    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:47.304960 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.304868    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67d0e76-4751-4e18-bb72-976c7e4c107b-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:47.304960 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.304876    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:47.304960 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:47.304884    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc57d\" (UniqueName: \"kubernetes.io/projected/e67d0e76-4751-4e18-bb72-976c7e4c107b-kube-api-access-jc57d\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:48.515176 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:48.515144    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" path="/var/lib/kubelet/pods/e67d0e76-4751-4e18-bb72-976c7e4c107b/volumes"
Sep 09 13:18:52.680128 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:52.680094    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"]
Sep 09 13:18:52.680660 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:52.680347    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg" podUID="ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" containerName="maas-api" containerID="cri-o://10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024" gracePeriod=30
Sep 09 13:18:52.744339 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:52.744306    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"]
Sep 09 13:18:52.744650 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:52.744576    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg" podUID="9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259" containerName="payload-pre-processing" containerID="cri-o://bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034" gracePeriod=30
Sep 09 13:18:53.008153 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.008126    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:18:53.039374 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.039350    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:18:53.048571 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.048550    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls\") pod \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") "
Sep 09 13:18:53.048681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.048582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-metrics-certs\") pod \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") "
Sep 09 13:18:53.048681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.048598    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bstzh\" (UniqueName: \"kubernetes.io/projected/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-kube-api-access-bstzh\") pod \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\" (UID: \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\") "
Sep 09 13:18:53.048681 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.048660    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-plugins-config-volume\") pod \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\" (UID: \"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259\") "
Sep 09 13:18:53.048855 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.048702    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84hls\" (UniqueName: \"kubernetes.io/projected/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-kube-api-access-84hls\") pod \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") "
Sep 09 13:18:53.048855 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.048754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-openshift-service-ca\") pod \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\" (UID: \"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67\") "
Sep 09 13:18:53.049227 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.049189    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259" (UID: "9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:18:53.049336 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.049222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" (UID: "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:18:53.050921 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.050899    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-kube-api-access-bstzh" (OuterVolumeSpecName: "kube-api-access-bstzh") pod "9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259" (UID: "9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259"). InnerVolumeSpecName "kube-api-access-bstzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:18:53.050921 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.050913    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" (UID: "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:18:53.051356 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.051325    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" (UID: "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:18:53.051356 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.051335    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-kube-api-access-84hls" (OuterVolumeSpecName: "kube-api-access-84hls") pod "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" (UID: "ae81c7bd-b21f-4a19-aa8d-92fbf0563f67"). InnerVolumeSpecName "kube-api-access-84hls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:18:53.150280 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.150254    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:53.150280 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.150277    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:53.150280 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.150287    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bstzh\" (UniqueName: \"kubernetes.io/projected/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-kube-api-access-bstzh\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:53.150478 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.150297    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:53.150478 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.150306    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84hls\" (UniqueName: \"kubernetes.io/projected/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-kube-api-access-84hls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:53.150478 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.150314    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:18:53.240178 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.240113    2577 generic.go:356] "Generic (PLEG): container finished" podID="9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259" containerID="bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034" exitCode=0
Sep 09 13:18:53.240178 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.240154    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"
Sep 09 13:18:53.240335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.240244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg" event={"ID":"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259","Type":"ContainerDied","Data":"bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034"}
Sep 09 13:18:53.240335 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.240306    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg" event={"ID":"9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259","Type":"ContainerDied","Data":"473bb1a528021471908443a9bdfb7d0e10012375b75629d16d91e2bb9affb486"}
Sep 09 13:18:53.240420 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.240342    2577 scope.go:117] "RemoveContainer" containerID="bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034"
Sep 09 13:18:53.248361 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.248334    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" containerID="10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024" exitCode=0
Sep 09 13:18:53.248470 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.248407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg" event={"ID":"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67","Type":"ContainerDied","Data":"10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024"}
Sep 09 13:18:53.248588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.248574    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"
Sep 09 13:18:53.248686 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.248567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg" event={"ID":"ae81c7bd-b21f-4a19-aa8d-92fbf0563f67","Type":"ContainerDied","Data":"b0f71bf79435d81bdd2de1a061279e0bc7c619085bdc01b8570e3e6298307b91"}
Sep 09 13:18:53.256084 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.256068    2577 scope.go:117] "RemoveContainer" containerID="bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034"
Sep 09 13:18:53.256355 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:18:53.256336    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034\": container with ID starting with bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034 not found: ID does not exist" containerID="bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034"
Sep 09 13:18:53.256396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.256364    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034"} err="failed to get container status \"bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034\": rpc error: code = NotFound desc = could not find container \"bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034\": container with ID starting with bea828c9b75d9d8a9836dd79989ee12ddf4f926c2b714fb1f0b08d663484d034 not found: ID does not exist"
Sep 09 13:18:53.256396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.256381    2577 scope.go:117] "RemoveContainer" containerID="10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024"
Sep 09 13:18:53.265030 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.265009    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"]
Sep 09 13:18:53.265313 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.265297    2577 scope.go:117] "RemoveContainer" containerID="10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024"
Sep 09 13:18:53.265606 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:18:53.265579    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024\": container with ID starting with 10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024 not found: ID does not exist" containerID="10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024"
Sep 09 13:18:53.265773 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.265673    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024"} err="failed to get container status \"10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024\": rpc error: code = NotFound desc = could not find container \"10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024\": container with ID starting with 10aebb0e8e0212455bc8020e3d730a711d6c2415be02dcc114092e8d6e385024 not found: ID does not exist"
Sep 09 13:18:53.267780 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.267756    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ada1ae-ddd64bc7d-sh2jg"]
Sep 09 13:18:53.278658 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.278622    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"]
Sep 09 13:18:53.283713 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:53.283695    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ada1ae-5758b5694d-vh4kg"]
Sep 09 13:18:54.515588 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:54.515555    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259" path="/var/lib/kubelet/pods/9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259/volumes"
Sep 09 13:18:54.516007 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:54.515951    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" path="/var/lib/kubelet/pods/ae81c7bd-b21f-4a19-aa8d-92fbf0563f67/volumes"
Sep 09 13:18:59.361769 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.361738    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"]
Sep 09 13:18:59.362155 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.361951    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy" containerID="cri-o://f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab" gracePeriod=30
Sep 09 13:18:59.731013 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.730972    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"]
Sep 09 13:18:59.731375 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731355    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731377    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731389    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731397    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731405    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731413    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731432    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0fa2864-cd6d-45d9-9829-89aa102d4e8e" containerName="payload-processing"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731441    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0fa2864-cd6d-45d9-9829-89aa102d4e8e" containerName="payload-processing"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731452    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy"
Sep 09 13:18:59.731460 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731461    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731470    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731478    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731487    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa219116-aacf-45e4-9909-79a6c8fa8816" containerName="maas-api"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731495    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa219116-aacf-45e4-9909-79a6c8fa8816" containerName="maas-api"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731513    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259" containerName="payload-pre-processing"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731522    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259" containerName="payload-pre-processing"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731532    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" containerName="maas-api"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731540    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" containerName="maas-api"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731652    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0fa2864-cd6d-45d9-9829-89aa102d4e8e" containerName="payload-processing"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731666    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731674    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa219116-aacf-45e4-9909-79a6c8fa8816" containerName="maas-api"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731685    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731692    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f1a66c9-4cfa-4c47-9a41-51b7c5a9a259" containerName="payload-pre-processing"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731700    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731708    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbfc0556-116e-4acf-94ae-95eeea61eff5" containerName="istio-proxy"
Sep 09 13:18:59.731923 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.731716    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae81c7bd-b21f-4a19-aa8d-92fbf0563f67" containerName="maas-api"
Sep 09 13:18:59.734723 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.734703    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"
Sep 09 13:18:59.736993 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.736974    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 13:18:59.737088 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.737050    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:18:59.742369 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.742150    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"]
Sep 09 13:18:59.796309 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.796285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4287f1-c558-476b-af43-10a67d1318af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk\" (UID: \"2a4287f1-c558-476b-af43-10a67d1318af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"
Sep 09 13:18:59.897110 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.897072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4287f1-c558-476b-af43-10a67d1318af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk\" (UID: \"2a4287f1-c558-476b-af43-10a67d1318af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"
Sep 09 13:18:59.897990 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:18:59.897958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4287f1-c558-476b-af43-10a67d1318af-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk\" (UID: \"2a4287f1-c558-476b-af43-10a67d1318af\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"
Sep 09 13:19:00.047900 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:00.047826    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"
Sep 09 13:19:00.315542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:00.315394    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"]
Sep 09 13:19:00.318221 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:19:00.318193    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4287f1_c558_476b_af43_10a67d1318af.slice/crio-eb8cb09cd4d3b485945f74e7baa3075c9184c55f9614f350383703a821654715 WatchSource:0}: Error finding container eb8cb09cd4d3b485945f74e7baa3075c9184c55f9614f350383703a821654715: Status 404 returned error can't find the container with id eb8cb09cd4d3b485945f74e7baa3075c9184c55f9614f350383703a821654715
Sep 09 13:19:01.288266 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:01.288177    2577 generic.go:356] "Generic (PLEG): container finished" podID="2a4287f1-c558-476b-af43-10a67d1318af" containerID="e8ffb30912cf374dde18bfbb6113ca98adfeaf4c24abaf100daef83a9beeb5c0" exitCode=0
Sep 09 13:19:01.288625 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:01.288266    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk" event={"ID":"2a4287f1-c558-476b-af43-10a67d1318af","Type":"ContainerDied","Data":"e8ffb30912cf374dde18bfbb6113ca98adfeaf4c24abaf100daef83a9beeb5c0"}
Sep 09 13:19:01.288625 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:01.288330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk" event={"ID":"2a4287f1-c558-476b-af43-10a67d1318af","Type":"ContainerStarted","Data":"eb8cb09cd4d3b485945f74e7baa3075c9184c55f9614f350383703a821654715"}
Sep 09 13:19:02.418245 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:02.418224    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"
Sep 09 13:19:02.517438 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:02.517404    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4287f1-c558-476b-af43-10a67d1318af-maas-api-service-ca\") pod \"2a4287f1-c558-476b-af43-10a67d1318af\" (UID: \"2a4287f1-c558-476b-af43-10a67d1318af\") "
Sep 09 13:19:02.517768 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:02.517749    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a4287f1-c558-476b-af43-10a67d1318af-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2a4287f1-c558-476b-af43-10a67d1318af" (UID: "2a4287f1-c558-476b-af43-10a67d1318af"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:19:02.614486 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:02.614423    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 13:19:02.614486 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:02.614473    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 13:19:02.618846 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:02.618824    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a4287f1-c558-476b-af43-10a67d1318af-maas-api-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:03.298540 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:03.298515    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"
Sep 09 13:19:03.298540 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:03.298535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk" event={"ID":"2a4287f1-c558-476b-af43-10a67d1318af","Type":"ContainerDied","Data":"eb8cb09cd4d3b485945f74e7baa3075c9184c55f9614f350383703a821654715"}
Sep 09 13:19:03.298793 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:03.298576    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb8cb09cd4d3b485945f74e7baa3075c9184c55f9614f350383703a821654715"
Sep 09 13:19:04.602212 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.602191    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:19:04.637135 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637114    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-data\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637279 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637163    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg6vg\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-kube-api-access-xg6vg\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637279 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637247    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f8514ef-8d13-479e-a124-c494753c35b8-istio-podinfo\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637279 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637271    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-credential-socket\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637456 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637300    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-certs\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637456 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637351    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-envoy\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637456 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:04.637456 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637389    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-istio-token\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637456 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637413    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f8514ef-8d13-479e-a124-c494753c35b8-istiod-ca-cert\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637456 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637446    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-socket\") pod \"5f8514ef-8d13-479e-a124-c494753c35b8\" (UID: \"5f8514ef-8d13-479e-a124-c494753c35b8\") "
Sep 09 13:19:04.637774 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637560    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:04.637774 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637575    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:04.637893 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637802    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:04.637893 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637827    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:04.637893 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637845    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:04.638051 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.637939    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:04.638131 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.638090    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f8514ef-8d13-479e-a124-c494753c35b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:19:04.639687 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.639668    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:19:04.639952 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.639935    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f8514ef-8d13-479e-a124-c494753c35b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:19:04.640078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.640055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:04.640078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.640070    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-kube-api-access-xg6vg" (OuterVolumeSpecName: "kube-api-access-xg6vg") pod "5f8514ef-8d13-479e-a124-c494753c35b8" (UID: "5f8514ef-8d13-479e-a124-c494753c35b8"). InnerVolumeSpecName "kube-api-access-xg6vg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:19:04.738377 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.738353    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg6vg\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-kube-api-access-xg6vg\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:04.738377 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.738374    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f8514ef-8d13-479e-a124-c494753c35b8-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:04.738508 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.738383    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:04.738508 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.738390    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f8514ef-8d13-479e-a124-c494753c35b8-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:04.738508 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.738397    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f8514ef-8d13-479e-a124-c494753c35b8-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:04.738508 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:04.738404    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f8514ef-8d13-479e-a124-c494753c35b8-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:05.309968 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.309929    2577 generic.go:356] "Generic (PLEG): container finished" podID="5f8514ef-8d13-479e-a124-c494753c35b8" containerID="f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab" exitCode=0
Sep 09 13:19:05.310140 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.309982    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"
Sep 09 13:19:05.310140 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.310018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" event={"ID":"5f8514ef-8d13-479e-a124-c494753c35b8","Type":"ContainerDied","Data":"f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab"}
Sep 09 13:19:05.310140 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.310074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458" event={"ID":"5f8514ef-8d13-479e-a124-c494753c35b8","Type":"ContainerDied","Data":"7c4247c26c3831d8facf582709d929934c62f085f75d341f18858a5e821d0914"}
Sep 09 13:19:05.310140 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.310101    2577 scope.go:117] "RemoveContainer" containerID="f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab"
Sep 09 13:19:05.319389 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.319372    2577 scope.go:117] "RemoveContainer" containerID="0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9"
Sep 09 13:19:05.331945 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.331917    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"]
Sep 09 13:19:05.332822 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.332806    2577 scope.go:117] "RemoveContainer" containerID="f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab"
Sep 09 13:19:05.333100 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:19:05.333083    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab\": container with ID starting with f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab not found: ID does not exist" containerID="f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab"
Sep 09 13:19:05.333156 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.333107    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab"} err="failed to get container status \"f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab\": rpc error: code = NotFound desc = could not find container \"f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab\": container with ID starting with f5c12bff95da88777a70212d7233ef13853dd7dd6f4022a743c8bea3ff8539ab not found: ID does not exist"
Sep 09 13:19:05.333156 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.333124    2577 scope.go:117] "RemoveContainer" containerID="0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9"
Sep 09 13:19:05.333353 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:19:05.333338    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9\": container with ID starting with 0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9 not found: ID does not exist" containerID="0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9"
Sep 09 13:19:05.333393 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.333358    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9"} err="failed to get container status \"0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9\": rpc error: code = NotFound desc = could not find container \"0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9\": container with ID starting with 0951a728329f2d9fd7bff4fe988ed95cb440f59620a863d940a7378272f3b1e9 not found: ID does not exist"
Sep 09 13:19:05.337027 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:05.337008    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ada1ae-openshift-default-5494b84d6b-dw458"]
Sep 09 13:19:06.516307 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:06.516271    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" path="/var/lib/kubelet/pods/5f8514ef-8d13-479e-a124-c494753c35b8/volumes"
Sep 09 13:19:10.224490 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.224461    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"]
Sep 09 13:19:10.225075 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.225042    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq" podUID="b00174d6-edfc-40eb-ad03-d8f13242eb07" containerName="payload-pre-processing" containerID="cri-o://2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6" gracePeriod=30
Sep 09 13:19:10.374177 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.374140    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"]
Sep 09 13:19:10.380158 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.380133    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-650a99-xgkzk"]
Sep 09 13:19:10.483973 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.483900    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:19:10.516343 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.516304    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a4287f1-c558-476b-af43-10a67d1318af" path="/var/lib/kubelet/pods/2a4287f1-c558-476b-af43-10a67d1318af/volumes"
Sep 09 13:19:10.582607 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.582580    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlsg7\" (UniqueName: \"kubernetes.io/projected/b00174d6-edfc-40eb-ad03-d8f13242eb07-kube-api-access-wlsg7\") pod \"b00174d6-edfc-40eb-ad03-d8f13242eb07\" (UID: \"b00174d6-edfc-40eb-ad03-d8f13242eb07\") "
Sep 09 13:19:10.582774 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.582627    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00174d6-edfc-40eb-ad03-d8f13242eb07-plugins-config-volume\") pod \"b00174d6-edfc-40eb-ad03-d8f13242eb07\" (UID: \"b00174d6-edfc-40eb-ad03-d8f13242eb07\") "
Sep 09 13:19:10.583010 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.582982    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00174d6-edfc-40eb-ad03-d8f13242eb07-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b00174d6-edfc-40eb-ad03-d8f13242eb07" (UID: "b00174d6-edfc-40eb-ad03-d8f13242eb07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:19:10.584859 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.584836    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b00174d6-edfc-40eb-ad03-d8f13242eb07-kube-api-access-wlsg7" (OuterVolumeSpecName: "kube-api-access-wlsg7") pod "b00174d6-edfc-40eb-ad03-d8f13242eb07" (UID: "b00174d6-edfc-40eb-ad03-d8f13242eb07"). InnerVolumeSpecName "kube-api-access-wlsg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:19:10.684127 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.684099    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlsg7\" (UniqueName: \"kubernetes.io/projected/b00174d6-edfc-40eb-ad03-d8f13242eb07-kube-api-access-wlsg7\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:10.684127 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:10.684124    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b00174d6-edfc-40eb-ad03-d8f13242eb07-plugins-config-volume\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:11.339597 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.339560    2577 generic.go:356] "Generic (PLEG): container finished" podID="b00174d6-edfc-40eb-ad03-d8f13242eb07" containerID="2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6" exitCode=0
Sep 09 13:19:11.340083 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.339609    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"
Sep 09 13:19:11.340083 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.339663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq" event={"ID":"b00174d6-edfc-40eb-ad03-d8f13242eb07","Type":"ContainerDied","Data":"2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6"}
Sep 09 13:19:11.340083 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.339715    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq" event={"ID":"b00174d6-edfc-40eb-ad03-d8f13242eb07","Type":"ContainerDied","Data":"9abcff00247d7ef9e52cc05743a594242e9d346996b54f4a9d23912930512f7d"}
Sep 09 13:19:11.340083 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.339737    2577 scope.go:117] "RemoveContainer" containerID="2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6"
Sep 09 13:19:11.348426 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.348411    2577 scope.go:117] "RemoveContainer" containerID="2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6"
Sep 09 13:19:11.348706 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:19:11.348687    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6\": container with ID starting with 2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6 not found: ID does not exist" containerID="2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6"
Sep 09 13:19:11.348773 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.348715    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6"} err="failed to get container status \"2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6\": rpc error: code = NotFound desc = could not find container \"2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6\": container with ID starting with 2afcf922aece7a2a0c128c21c748cc33e975bc73820c84cd1e3435b77ddc1be6 not found: ID does not exist"
Sep 09 13:19:11.363078 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.363059    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"]
Sep 09 13:19:11.387365 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:11.387342    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-650a99-6645df486-lnvpq"]
Sep 09 13:19:12.515235 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:12.515200    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b00174d6-edfc-40eb-ad03-d8f13242eb07" path="/var/lib/kubelet/pods/b00174d6-edfc-40eb-ad03-d8f13242eb07/volumes"
Sep 09 13:19:16.996997 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:16.996957    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"]
Sep 09 13:19:16.997480 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:16.997168    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" containerID="cri-o://52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83" gracePeriod=30
Sep 09 13:19:20.845985 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:20.845951    2577 patch_prober.go:28] interesting pod/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 13:19:20.846338 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:20.846006    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 13:19:22.257624 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.257603    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:19:22.377909 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.377884    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-token\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.377936    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istiod-ca-cert\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.377959    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-data\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378073 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.377978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-socket\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378265 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378088    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-certs\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378265 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378122    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2ckn\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-kube-api-access-t2ckn\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378265 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378151    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-podinfo\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378265 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378231    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-envoy\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378488 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378266    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-credential-socket\") pod \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\" (UID: \"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e\") "
Sep 09 13:19:22.378488 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378302    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:22.378488 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378381    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:19:22.378691 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378559    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istiod-ca-cert\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.378691 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378565    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:22.378691 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378578    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.378691 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378596    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:22.378987 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.378963    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:22.381013 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.380958    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:19:22.381299 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.381266    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:19:22.381390 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.381304    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-kube-api-access-t2ckn" (OuterVolumeSpecName: "kube-api-access-t2ckn") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "kube-api-access-t2ckn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:19:22.381550 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.381529    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" (UID: "ff8293f6-36cd-4fa4-bbf8-23a7f30d092e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:19:22.394754 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.394726    2577 generic.go:356] "Generic (PLEG): container finished" podID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerID="52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83" exitCode=0
Sep 09 13:19:22.394874 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.394789    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"
Sep 09 13:19:22.394874 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.394798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" event={"ID":"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e","Type":"ContainerDied","Data":"52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83"}
Sep 09 13:19:22.394874 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.394856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf" event={"ID":"ff8293f6-36cd-4fa4-bbf8-23a7f30d092e","Type":"ContainerDied","Data":"a8fdcb987cf43b5ff5f20b9524fde9109392a6dd2904bb4784e54ecaa8e4b1fc"}
Sep 09 13:19:22.395027 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.394890    2577 scope.go:117] "RemoveContainer" containerID="52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83"
Sep 09 13:19:22.405419 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.405399    2577 scope.go:117] "RemoveContainer" containerID="3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e"
Sep 09 13:19:22.415989 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.415603    2577 scope.go:117] "RemoveContainer" containerID="52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83"
Sep 09 13:19:22.415989 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:19:22.415952    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83\": container with ID starting with 52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83 not found: ID does not exist" containerID="52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83"
Sep 09 13:19:22.416121 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.415986    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83"} err="failed to get container status \"52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83\": rpc error: code = NotFound desc = could not find container \"52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83\": container with ID starting with 52e4715800286b92cd12936cba33956fb05a9000bf8149d7fc197e24dd97fa83 not found: ID does not exist"
Sep 09 13:19:22.416121 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.416008    2577 scope.go:117] "RemoveContainer" containerID="3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e"
Sep 09 13:19:22.417125 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:19:22.417031    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e\": container with ID starting with 3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e not found: ID does not exist" containerID="3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e"
Sep 09 13:19:22.417125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.417083    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e"} err="failed to get container status \"3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e\": rpc error: code = NotFound desc = could not find container \"3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e\": container with ID starting with 3700086d93d92b638da3b5a86d08a7ee38f99489c60e096312443d728758801e not found: ID does not exist"
Sep 09 13:19:22.418529 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.418482    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"]
Sep 09 13:19:22.422165 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.422143    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-650a99-openshift-default-b867b8bc4-4cjjf"]
Sep 09 13:19:22.479323 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.479300    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-token\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.479323 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.479321    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-data\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.479475 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.479330    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-workload-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.479475 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.479338    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2ckn\" (UniqueName: \"kubernetes.io/projected/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-kube-api-access-t2ckn\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.479475 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.479347    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-podinfo\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.479475 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.479355    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-istio-envoy\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.479475 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.479363    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e-credential-socket\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:19:22.515252 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:19:22.515219    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" path="/var/lib/kubelet/pods/ff8293f6-36cd-4fa4-bbf8-23a7f30d092e/volumes"
Sep 09 13:23:35.663980 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.663942    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"]
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664225    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664236    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664244    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664250    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664257    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a4287f1-c558-476b-af43-10a67d1318af" containerName="revoke-keys"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664264    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a4287f1-c558-476b-af43-10a67d1318af" containerName="revoke-keys"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664273    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664278    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664293    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b00174d6-edfc-40eb-ad03-d8f13242eb07" containerName="payload-pre-processing"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664298    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b00174d6-edfc-40eb-ad03-d8f13242eb07" containerName="payload-pre-processing"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664342    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a4287f1-c558-476b-af43-10a67d1318af" containerName="revoke-keys"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664349    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67d0e76-4751-4e18-bb72-976c7e4c107b" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664356    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664361    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b00174d6-edfc-40eb-ad03-d8f13242eb07" containerName="payload-pre-processing"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664367    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664372    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664376    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="76b8304a-7f81-40f8-aa68-de9b171547f8" containerName="istio-proxy"
Sep 09 13:23:35.664396 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.664381    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy"
Sep 09 13:23:35.667190 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.667172    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.669589 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.669570    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-947aaaa2-kserve-self-signed-certs\""
Sep 09 13:23:35.678300 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.678269    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"]
Sep 09 13:23:35.856714 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.856685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-model-cache\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.856905 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.856734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-766xb\" (UniqueName: \"kubernetes.io/projected/89364443-bdcf-4c73-91d8-22766f1cc7c4-kube-api-access-766xb\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.856905 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.856796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-dshm\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.856905 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.856832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-tmp-dir\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.856905 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.856859    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-home\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.856905 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.856875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89364443-bdcf-4c73-91d8-22766f1cc7c4-tls-certs\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.957747 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.957669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-766xb\" (UniqueName: \"kubernetes.io/projected/89364443-bdcf-4c73-91d8-22766f1cc7c4-kube-api-access-766xb\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.957747 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.957726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-dshm\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.957953 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.957770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-tmp-dir\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.957953 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.957805    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-home\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.957953 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.957829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89364443-bdcf-4c73-91d8-22766f1cc7c4-tls-certs\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.957953 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.957868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-model-cache\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.958246 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.958219    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-tmp-dir\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.958381 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.958268    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-home\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.958381 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.958313    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-model-cache\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.960388 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.960362    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-dshm\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.961095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.961036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89364443-bdcf-4c73-91d8-22766f1cc7c4-tls-certs\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.968739 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.968718    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-766xb\" (UniqueName: \"kubernetes.io/projected/89364443-bdcf-4c73-91d8-22766f1cc7c4-kube-api-access-766xb\") pod \"e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") " pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:35.975511 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:35.975486    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:36.248428 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:36.248402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"]
Sep 09 13:23:36.250055 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:23:36.250022    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89364443_bdcf_4c73_91d8_22766f1cc7c4.slice/crio-c48b210917061f17d029942ddfd6bfe0b4771259fa89a3f246e614d10f329d5b WatchSource:0}: Error finding container c48b210917061f17d029942ddfd6bfe0b4771259fa89a3f246e614d10f329d5b: Status 404 returned error can't find the container with id c48b210917061f17d029942ddfd6bfe0b4771259fa89a3f246e614d10f329d5b
Sep 09 13:23:36.515824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:36.515738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5" event={"ID":"89364443-bdcf-4c73-91d8-22766f1cc7c4","Type":"ContainerStarted","Data":"c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261"}
Sep 09 13:23:36.515824 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:36.515773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5" event={"ID":"89364443-bdcf-4c73-91d8-22766f1cc7c4","Type":"ContainerStarted","Data":"c48b210917061f17d029942ddfd6bfe0b4771259fa89a3f246e614d10f329d5b"}
Sep 09 13:23:36.538306 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:36.538257    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5" podStartSLOduration=1.538241176 podStartE2EDuration="1.538241176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:23:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:23:36.537329105 +0000 UTC m=+2102.537995217" watchObservedRunningTime="2026-09-09 13:23:36.538241176 +0000 UTC m=+2102.538907276"
Sep 09 13:23:45.976531 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:45.976494    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:45.977007 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:45.976546    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:45.989045 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:45.989022    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:46.570114 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:46.570081    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:23:59.052761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:23:59.052727    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"]
Sep 09 13:24:14.072958 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.072916    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5" podUID="89364443-bdcf-4c73-91d8-22766f1cc7c4" containerName="main" containerID="cri-o://c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261" gracePeriod=45
Sep 09 13:24:14.314164 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.314143    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:24:14.469850 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.469822    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-home\") pod \"89364443-bdcf-4c73-91d8-22766f1cc7c4\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") "
Sep 09 13:24:14.469981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.469858    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89364443-bdcf-4c73-91d8-22766f1cc7c4-tls-certs\") pod \"89364443-bdcf-4c73-91d8-22766f1cc7c4\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") "
Sep 09 13:24:14.469981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.469883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-766xb\" (UniqueName: \"kubernetes.io/projected/89364443-bdcf-4c73-91d8-22766f1cc7c4-kube-api-access-766xb\") pod \"89364443-bdcf-4c73-91d8-22766f1cc7c4\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") "
Sep 09 13:24:14.469981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.469913    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-tmp-dir\") pod \"89364443-bdcf-4c73-91d8-22766f1cc7c4\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") "
Sep 09 13:24:14.470126 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.470004    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-model-cache\") pod \"89364443-bdcf-4c73-91d8-22766f1cc7c4\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") "
Sep 09 13:24:14.470126 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.470029    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-dshm\") pod \"89364443-bdcf-4c73-91d8-22766f1cc7c4\" (UID: \"89364443-bdcf-4c73-91d8-22766f1cc7c4\") "
Sep 09 13:24:14.470126 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.470101    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-home" (OuterVolumeSpecName: "home") pod "89364443-bdcf-4c73-91d8-22766f1cc7c4" (UID: "89364443-bdcf-4c73-91d8-22766f1cc7c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:24:14.470291 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.470184    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "89364443-bdcf-4c73-91d8-22766f1cc7c4" (UID: "89364443-bdcf-4c73-91d8-22766f1cc7c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:24:14.470349 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.470304    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-home\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:14.470349 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.470303    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-model-cache" (OuterVolumeSpecName: "model-cache") pod "89364443-bdcf-4c73-91d8-22766f1cc7c4" (UID: "89364443-bdcf-4c73-91d8-22766f1cc7c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:24:14.470349 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.470319    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-tmp-dir\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:14.472233 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.472213    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89364443-bdcf-4c73-91d8-22766f1cc7c4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "89364443-bdcf-4c73-91d8-22766f1cc7c4" (UID: "89364443-bdcf-4c73-91d8-22766f1cc7c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:24:14.472233 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.472226    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-dshm" (OuterVolumeSpecName: "dshm") pod "89364443-bdcf-4c73-91d8-22766f1cc7c4" (UID: "89364443-bdcf-4c73-91d8-22766f1cc7c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:24:14.472352 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.472249    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89364443-bdcf-4c73-91d8-22766f1cc7c4-kube-api-access-766xb" (OuterVolumeSpecName: "kube-api-access-766xb") pod "89364443-bdcf-4c73-91d8-22766f1cc7c4" (UID: "89364443-bdcf-4c73-91d8-22766f1cc7c4"). InnerVolumeSpecName "kube-api-access-766xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:24:14.571466 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.571445    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-766xb\" (UniqueName: \"kubernetes.io/projected/89364443-bdcf-4c73-91d8-22766f1cc7c4-kube-api-access-766xb\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:14.571466 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.571464    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-model-cache\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:14.571601 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.571473    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89364443-bdcf-4c73-91d8-22766f1cc7c4-dshm\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:14.571601 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.571481    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89364443-bdcf-4c73-91d8-22766f1cc7c4-tls-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:14.685677 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.685627    2577 generic.go:356] "Generic (PLEG): container finished" podID="89364443-bdcf-4c73-91d8-22766f1cc7c4" containerID="c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261" exitCode=0
Sep 09 13:24:14.685771 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.685693    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"
Sep 09 13:24:14.685808 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.685695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5" event={"ID":"89364443-bdcf-4c73-91d8-22766f1cc7c4","Type":"ContainerDied","Data":"c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261"}
Sep 09 13:24:14.685848 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.685815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5" event={"ID":"89364443-bdcf-4c73-91d8-22766f1cc7c4","Type":"ContainerDied","Data":"c48b210917061f17d029942ddfd6bfe0b4771259fa89a3f246e614d10f329d5b"}
Sep 09 13:24:14.685848 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.685838    2577 scope.go:117] "RemoveContainer" containerID="c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261"
Sep 09 13:24:14.694026 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.694011    2577 scope.go:117] "RemoveContainer" containerID="c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261"
Sep 09 13:24:14.694316 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:24:14.694299    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261\": container with ID starting with c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261 not found: ID does not exist" containerID="c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261"
Sep 09 13:24:14.694364 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.694324    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261"} err="failed to get container status \"c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261\": rpc error: code = NotFound desc = could not find container \"c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261\": container with ID starting with c71dbc4fdea94f026764a7d219e25d9bdad9391bca2481868ad290d4cd4ef261 not found: ID does not exist"
Sep 09 13:24:14.701506 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.701483    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"]
Sep 09 13:24:14.704455 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:14.704437    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-947aaaa2-kserve-d564849bc-nsll5"]
Sep 09 13:24:15.266132 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.266095    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"]
Sep 09 13:24:15.266761 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.266728    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy"
Sep 09 13:24:15.266831 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.266764    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff8293f6-36cd-4fa4-bbf8-23a7f30d092e" containerName="istio-proxy"
Sep 09 13:24:15.266831 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.266800    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89364443-bdcf-4c73-91d8-22766f1cc7c4" containerName="main"
Sep 09 13:24:15.266831 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.266808    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="89364443-bdcf-4c73-91d8-22766f1cc7c4" containerName="main"
Sep 09 13:24:15.266962 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.266838    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy"
Sep 09 13:24:15.266962 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.266846    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f8514ef-8d13-479e-a124-c494753c35b8" containerName="istio-proxy"
Sep 09 13:24:15.267039 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.266989    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="89364443-bdcf-4c73-91d8-22766f1cc7c4" containerName="main"
Sep 09 13:24:15.271784 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.271758    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.275611 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.275587    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 13:24:15.275870 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.275837    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 13:24:15.275870 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.275837    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dbjk4\""
Sep 09 13:24:15.275870 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.275905    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 13:24:15.276125 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.276004    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 13:24:15.279158 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.279138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"]
Sep 09 13:24:15.286373 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.286353    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"]
Sep 09 13:24:15.289505 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.289488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.302299 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.302277    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"]
Sep 09 13:24:15.376512 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.376477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trcgs\" (UniqueName: \"kubernetes.io/projected/cf6e8893-f5bf-4328-ab34-26feefb55092-kube-api-access-trcgs\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.376707 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.376519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-maas-api-tls\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.376707 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.376554    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a0d57c2-9f62-4b7e-a776-928880c58c4b-maas-api-tls\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.376707 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.376617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6e8893-f5bf-4328-ab34-26feefb55092-openshift-service-ca\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.376707 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.376671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0d57c2-9f62-4b7e-a776-928880c58c4b-openshift-service-ca\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.376707 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.376704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a0d57c2-9f62-4b7e-a776-928880c58c4b-metrics-certs\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.376936 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.376770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gnrx\" (UniqueName: \"kubernetes.io/projected/2a0d57c2-9f62-4b7e-a776-928880c58c4b-kube-api-access-5gnrx\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.376936 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.376801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-metrics-certs\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.477408 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.477374    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a0d57c2-9f62-4b7e-a776-928880c58c4b-maas-api-tls\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.477542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.477421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6e8893-f5bf-4328-ab34-26feefb55092-openshift-service-ca\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.477542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.477438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0d57c2-9f62-4b7e-a776-928880c58c4b-openshift-service-ca\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.477542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.477457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a0d57c2-9f62-4b7e-a776-928880c58c4b-metrics-certs\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.477542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.477486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gnrx\" (UniqueName: \"kubernetes.io/projected/2a0d57c2-9f62-4b7e-a776-928880c58c4b-kube-api-access-5gnrx\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.477542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.477505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-metrics-certs\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.477542 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.477526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trcgs\" (UniqueName: \"kubernetes.io/projected/cf6e8893-f5bf-4328-ab34-26feefb55092-kube-api-access-trcgs\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.477873 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.477547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-maas-api-tls\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.478263 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.478239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6e8893-f5bf-4328-ab34-26feefb55092-openshift-service-ca\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.478372 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.478239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0d57c2-9f62-4b7e-a776-928880c58c4b-openshift-service-ca\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.480097 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.480075    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a0d57c2-9f62-4b7e-a776-928880c58c4b-maas-api-tls\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.480193 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.480150    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-metrics-certs\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.480424 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.480408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a0d57c2-9f62-4b7e-a776-928880c58c4b-metrics-certs\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.480480 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.480415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-maas-api-tls\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.485647 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.485606    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trcgs\" (UniqueName: \"kubernetes.io/projected/cf6e8893-f5bf-4328-ab34-26feefb55092-kube-api-access-trcgs\") pod \"maas-api-5dc88f4c96-pf4sf\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.486286 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.486267    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gnrx\" (UniqueName: \"kubernetes.io/projected/2a0d57c2-9f62-4b7e-a776-928880c58c4b-kube-api-access-5gnrx\") pod \"maas-api-6b9c84785b-v4wj6\" (UID: \"2a0d57c2-9f62-4b7e-a776-928880c58c4b\") " pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.585920 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.585836    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:15.599933 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.599904    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:15.861012 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.860987    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"]
Sep 09 13:24:15.863086 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:24:15.863057    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6e8893_f5bf_4328_ab34_26feefb55092.slice/crio-367e175525e6de75d1f1fdc5f07244f0624b594976d8f90f7f6840a12b0ed70f WatchSource:0}: Error finding container 367e175525e6de75d1f1fdc5f07244f0624b594976d8f90f7f6840a12b0ed70f: Status 404 returned error can't find the container with id 367e175525e6de75d1f1fdc5f07244f0624b594976d8f90f7f6840a12b0ed70f
Sep 09 13:24:15.864433 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.864411    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:24:15.880823 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:15.880799    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"]
Sep 09 13:24:15.883109 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:24:15.883087    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0d57c2_9f62_4b7e_a776_928880c58c4b.slice/crio-a3c625aa5fe56d95742357ab6cb79c82784589baf7a1d7de14af53251ddf8979 WatchSource:0}: Error finding container a3c625aa5fe56d95742357ab6cb79c82784589baf7a1d7de14af53251ddf8979: Status 404 returned error can't find the container with id a3c625aa5fe56d95742357ab6cb79c82784589baf7a1d7de14af53251ddf8979
Sep 09 13:24:16.516296 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.516264    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89364443-bdcf-4c73-91d8-22766f1cc7c4" path="/var/lib/kubelet/pods/89364443-bdcf-4c73-91d8-22766f1cc7c4/volumes"
Sep 09 13:24:16.700046 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.699936    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf" event={"ID":"cf6e8893-f5bf-4328-ab34-26feefb55092","Type":"ContainerStarted","Data":"585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a"}
Sep 09 13:24:16.700046 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.700005    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf" event={"ID":"cf6e8893-f5bf-4328-ab34-26feefb55092","Type":"ContainerStarted","Data":"367e175525e6de75d1f1fdc5f07244f0624b594976d8f90f7f6840a12b0ed70f"}
Sep 09 13:24:16.700255 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.700122    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:16.701614 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.701590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6" event={"ID":"2a0d57c2-9f62-4b7e-a776-928880c58c4b","Type":"ContainerStarted","Data":"231e6252bad852bce755c1be715f10e3341836886e4911801ee726ae967f7f39"}
Sep 09 13:24:16.701753 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.701661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6" event={"ID":"2a0d57c2-9f62-4b7e-a776-928880c58c4b","Type":"ContainerStarted","Data":"a3c625aa5fe56d95742357ab6cb79c82784589baf7a1d7de14af53251ddf8979"}
Sep 09 13:24:16.701753 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.701706    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:16.717191 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.717133    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf" podStartSLOduration=1.131606339 podStartE2EDuration="1.717113276s" totalImagesPullingTime="585.506937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:24:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:24:16.716257944 +0000 UTC m=+2142.716924044" watchObservedRunningTime="2026-09-09 13:24:16.717113276 +0000 UTC m=+2142.717779375"
Sep 09 13:24:16.734816 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:16.734765    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6" podStartSLOduration=1.1274331499999999 podStartE2EDuration="1.734750189s" totalImagesPullingTime="607.317039ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:24:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:24:16.73381268 +0000 UTC m=+2142.734478804" watchObservedRunningTime="2026-09-09 13:24:16.734750189 +0000 UTC m=+2142.735416288"
Sep 09 13:24:22.713238 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:22.713208    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b9c84785b-v4wj6"
Sep 09 13:24:22.713752 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:22.713286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:22.775744 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:22.775711    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"]
Sep 09 13:24:22.775912 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:22.775887    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf" podUID="cf6e8893-f5bf-4328-ab34-26feefb55092" containerName="maas-api" containerID="cri-o://585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a" gracePeriod=30
Sep 09 13:24:23.012097 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.012074    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:23.151828 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.151800    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6e8893-f5bf-4328-ab34-26feefb55092-openshift-service-ca\") pod \"cf6e8893-f5bf-4328-ab34-26feefb55092\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") "
Sep 09 13:24:23.151981 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.151883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-maas-api-tls\") pod \"cf6e8893-f5bf-4328-ab34-26feefb55092\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") "
Sep 09 13:24:23.152052 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.152033    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-metrics-certs\") pod \"cf6e8893-f5bf-4328-ab34-26feefb55092\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") "
Sep 09 13:24:23.152095 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.152073    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trcgs\" (UniqueName: \"kubernetes.io/projected/cf6e8893-f5bf-4328-ab34-26feefb55092-kube-api-access-trcgs\") pod \"cf6e8893-f5bf-4328-ab34-26feefb55092\" (UID: \"cf6e8893-f5bf-4328-ab34-26feefb55092\") "
Sep 09 13:24:23.152167 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.152149    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf6e8893-f5bf-4328-ab34-26feefb55092-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf6e8893-f5bf-4328-ab34-26feefb55092" (UID: "cf6e8893-f5bf-4328-ab34-26feefb55092"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:24:23.152371 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.152351    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6e8893-f5bf-4328-ab34-26feefb55092-openshift-service-ca\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:23.154085 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.154063    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf6e8893-f5bf-4328-ab34-26feefb55092" (UID: "cf6e8893-f5bf-4328-ab34-26feefb55092"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:24:23.154197 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.154143    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cf6e8893-f5bf-4328-ab34-26feefb55092" (UID: "cf6e8893-f5bf-4328-ab34-26feefb55092"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:24:23.154197 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.154166    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf6e8893-f5bf-4328-ab34-26feefb55092-kube-api-access-trcgs" (OuterVolumeSpecName: "kube-api-access-trcgs") pod "cf6e8893-f5bf-4328-ab34-26feefb55092" (UID: "cf6e8893-f5bf-4328-ab34-26feefb55092"). InnerVolumeSpecName "kube-api-access-trcgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:24:23.253102 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.253031    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-maas-api-tls\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:23.253102 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.253053    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf6e8893-f5bf-4328-ab34-26feefb55092-metrics-certs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:23.253102 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.253062    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trcgs\" (UniqueName: \"kubernetes.io/projected/cf6e8893-f5bf-4328-ab34-26feefb55092-kube-api-access-trcgs\") on node \"ip-10-0-132-160.ec2.internal\" DevicePath \"\""
Sep 09 13:24:23.738370 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.738331    2577 generic.go:356] "Generic (PLEG): container finished" podID="cf6e8893-f5bf-4328-ab34-26feefb55092" containerID="585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a" exitCode=0
Sep 09 13:24:23.738898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.738418    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"
Sep 09 13:24:23.738898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.738413    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf" event={"ID":"cf6e8893-f5bf-4328-ab34-26feefb55092","Type":"ContainerDied","Data":"585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a"}
Sep 09 13:24:23.738898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.738575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf" event={"ID":"cf6e8893-f5bf-4328-ab34-26feefb55092","Type":"ContainerDied","Data":"367e175525e6de75d1f1fdc5f07244f0624b594976d8f90f7f6840a12b0ed70f"}
Sep 09 13:24:23.738898 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.738615    2577 scope.go:117] "RemoveContainer" containerID="585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a"
Sep 09 13:24:23.749732 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.749671    2577 scope.go:117] "RemoveContainer" containerID="585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a"
Sep 09 13:24:23.750128 ip-10-0-132-160 kubenswrapper[2577]: E0909 13:24:23.750096    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a\": container with ID starting with 585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a not found: ID does not exist" containerID="585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a"
Sep 09 13:24:23.750229 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.750137    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a"} err="failed to get container status \"585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a\": rpc error: code = NotFound desc = could not find container \"585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a\": container with ID starting with 585170f5abd72e5e132e0ff260a2b6dc63c900d917f6c818e9728b2ce1383c7a not found: ID does not exist"
Sep 09 13:24:23.762423 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.762395    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"]
Sep 09 13:24:23.764191 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:23.764167    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-pf4sf"]
Sep 09 13:24:24.515090 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:24.515057    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf6e8893-f5bf-4328-ab34-26feefb55092" path="/var/lib/kubelet/pods/cf6e8893-f5bf-4328-ab34-26feefb55092/volumes"
Sep 09 13:24:51.593248 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:51.593217    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pcl9t_ae1f072f-f462-4b0e-b81e-4624c675bad9/server/0.log"
Sep 09 13:24:51.730731 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:51.730701    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j7lmg_110e7393-0f60-4b1a-b58f-62f216970c24/manager/0.log"
Sep 09 13:24:51.988784 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:51.988754    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647774d94-9jldd_f8b024cc-fea3-4dcc-b7d7-73b48fc94288/copy-manifests/0.log"
Sep 09 13:24:51.996071 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:51.996037    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647774d94-9jldd_f8b024cc-fea3-4dcc-b7d7-73b48fc94288/manager/0.log"
Sep 09 13:24:52.610577 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:52.610543    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6b9c84785b-v4wj6_2a0d57c2-9f62-4b7e-a776-928880c58c4b/maas-api/0.log"
Sep 09 13:24:52.854564 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:52.854540    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29815995-vlc66_6c2635f9-5c15-4a7f-a991-b73ba5fa0d2f/cleanup/0.log"
Sep 09 13:24:54.929529 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:54.929496    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cb5blq_5d270b03-da4f-49e1-932c-5bf1304ee88e/istio-proxy/0.log"
Sep 09 13:24:55.060394 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:55.060352    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hdh5c_3d2e3c5e-df61-41b0-b145-a1c4d245f844/discovery/0.log"
Sep 09 13:24:55.416427 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:55.416388    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rfh8m_5bfd0059-dde3-45f8-8570-612cfdd5a1fc/istio-proxy/1.log"
Sep 09 13:24:55.551002 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:55.550954    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gm2kv_7368e50b-ec89-4339-a32c-599551b9e8a2/payload-pre-processing/0.log"
Sep 09 13:24:56.161727 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:24:56.161690    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wfsrz_6eb7bb09-fd11-4cbe-a834-ad287c5a24d7/main/0.log"
Sep 09 13:25:06.049100 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:06.049047    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wvkhv_d2e18a6d-9313-4b12-8109-61a4d930b9e3/global-pull-secret-syncer/0.log"
Sep 09 13:25:06.176582 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:06.176550    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pnkgb_fa65dcab-8e63-4217-8118-0ae75f5dd140/konnectivity-agent/0.log"
Sep 09 13:25:06.250441 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:06.250409    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-160.ec2.internal_e58beafae37c3742927ee4ffcc21b4b4/haproxy/0.log"
Sep 09 13:25:12.744192 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:12.744162    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5f7d488c4c-n5pfw_77e98066-88f4-43cd-9bff-3e40850d6d1e/metrics-server/0.log"
Sep 09 13:25:12.954851 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:12.954816    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwg9d_9fa88c30-3155-49b8-a3e6-fce37598f630/node-exporter/0.log"
Sep 09 13:25:12.997676 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:12.997580    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwg9d_9fa88c30-3155-49b8-a3e6-fce37598f630/kube-rbac-proxy/0.log"
Sep 09 13:25:13.022928 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:13.022902    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hwg9d_9fa88c30-3155-49b8-a3e6-fce37598f630/init-textfile/0.log"
Sep 09 13:25:15.593291 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.593254    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"]
Sep 09 13:25:15.593731 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.593548    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6e8893-f5bf-4328-ab34-26feefb55092" containerName="maas-api"
Sep 09 13:25:15.593731 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.593559    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6e8893-f5bf-4328-ab34-26feefb55092" containerName="maas-api"
Sep 09 13:25:15.593731 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.593612    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6e8893-f5bf-4328-ab34-26feefb55092" containerName="maas-api"
Sep 09 13:25:15.596562 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.596542    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.598918 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.598871    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sv79w\"/\"openshift-service-ca.crt\""
Sep 09 13:25:15.599050 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.599034    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sv79w\"/\"default-dockercfg-d6hxv\""
Sep 09 13:25:15.599119 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.599105    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sv79w\"/\"kube-root-ca.crt\""
Sep 09 13:25:15.610264 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.610244    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"]
Sep 09 13:25:15.712687 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.712661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-proc\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.712827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.712692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-lib-modules\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.712827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.712757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftxm9\" (UniqueName: \"kubernetes.io/projected/75d4f399-57d2-4cd2-9dd6-2c9cce196143-kube-api-access-ftxm9\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.712827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.712782    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-sys\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.712827 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.712805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-podres\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.813652 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-proc\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.813652 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813657    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-lib-modules\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.813847 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftxm9\" (UniqueName: \"kubernetes.io/projected/75d4f399-57d2-4cd2-9dd6-2c9cce196143-kube-api-access-ftxm9\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.813847 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813744    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-proc\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.813847 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-lib-modules\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.813847 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-sys\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.813847 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813839    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-podres\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.813847 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813842    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-sys\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.814046 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.813921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75d4f399-57d2-4cd2-9dd6-2c9cce196143-podres\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.823087 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.823062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftxm9\" (UniqueName: \"kubernetes.io/projected/75d4f399-57d2-4cd2-9dd6-2c9cce196143-kube-api-access-ftxm9\") pod \"perf-node-gather-daemonset-vbznh\" (UID: \"75d4f399-57d2-4cd2-9dd6-2c9cce196143\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:15.906780 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:15.906757    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:16.177490 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:16.177462    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"]
Sep 09 13:25:16.180142 ip-10-0-132-160 kubenswrapper[2577]: W0909 13:25:16.180101    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod75d4f399_57d2_4cd2_9dd6_2c9cce196143.slice/crio-c75f94288be7dffb8c4e48354fe274bf95081da24ed9e8dd188329562ec9e138 WatchSource:0}: Error finding container c75f94288be7dffb8c4e48354fe274bf95081da24ed9e8dd188329562ec9e138: Status 404 returned error can't find the container with id c75f94288be7dffb8c4e48354fe274bf95081da24ed9e8dd188329562ec9e138
Sep 09 13:25:16.976233 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:16.976198    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh" event={"ID":"75d4f399-57d2-4cd2-9dd6-2c9cce196143","Type":"ContainerStarted","Data":"b2ae85d169140b14992e508efb3661d4f0c5be0bdf1de5cc59628a8c695d47de"}
Sep 09 13:25:16.976759 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:16.976259    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh" event={"ID":"75d4f399-57d2-4cd2-9dd6-2c9cce196143","Type":"ContainerStarted","Data":"c75f94288be7dffb8c4e48354fe274bf95081da24ed9e8dd188329562ec9e138"}
Sep 09 13:25:16.976759 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:16.976340    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:16.995887 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:16.995846    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh" podStartSLOduration=1.9958318959999999 podStartE2EDuration="1.995831896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:25:16.993702803 +0000 UTC m=+2202.994368902" watchObservedRunningTime="2026-09-09 13:25:16.995831896 +0000 UTC m=+2202.996497995"
Sep 09 13:25:18.402675 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:18.402618    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fkxwl_c7412b12-e83b-4c9a-a98a-bff21efb9dee/dns/0.log"
Sep 09 13:25:18.430129 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:18.430103    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fkxwl_c7412b12-e83b-4c9a-a98a-bff21efb9dee/kube-rbac-proxy/0.log"
Sep 09 13:25:18.598056 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:18.598025    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dgkg9_482bfe86-6cb5-4c04-a941-e3e81ff24ad6/dns-node-resolver/0.log"
Sep 09 13:25:19.214984 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:19.214959    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2cjcv_bbad6c3c-19dc-4d5c-a127-7f47350d618a/node-ca/0.log"
Sep 09 13:25:20.249928 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:20.249893    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cb5blq_5d270b03-da4f-49e1-932c-5bf1304ee88e/istio-proxy/0.log"
Sep 09 13:25:20.658511 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:20.658485    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hdh5c_3d2e3c5e-df61-41b0-b145-a1c4d245f844/discovery/0.log"
Sep 09 13:25:20.760544 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:20.760504    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rfh8m_5bfd0059-dde3-45f8-8570-612cfdd5a1fc/istio-proxy/0.log"
Sep 09 13:25:20.776098 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:20.776073    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rfh8m_5bfd0059-dde3-45f8-8570-612cfdd5a1fc/istio-proxy/1.log"
Sep 09 13:25:20.829424 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:20.829402    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gm2kv_7368e50b-ec89-4339-a32c-599551b9e8a2/payload-pre-processing/0.log"
Sep 09 13:25:21.868308 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:21.868267    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4jwks_4ee4dc29-0e0c-4f78-93b0-73c254525239/serve-healthcheck-canary/0.log"
Sep 09 13:25:22.439506 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:22.439474    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dqhj7_e04d3a42-5b3e-4570-8c33-0f8c54eda3f2/kube-rbac-proxy/0.log"
Sep 09 13:25:22.464971 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:22.464951    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dqhj7_e04d3a42-5b3e-4570-8c33-0f8c54eda3f2/exporter/0.log"
Sep 09 13:25:22.498815 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:22.498794    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dqhj7_e04d3a42-5b3e-4570-8c33-0f8c54eda3f2/extractor/0.log"
Sep 09 13:25:22.990895 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:22.990866    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-vbznh"
Sep 09 13:25:25.815695 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:25.815665    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-pcl9t_ae1f072f-f462-4b0e-b81e-4624c675bad9/server/0.log"
Sep 09 13:25:25.943601 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:25.943576    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j7lmg_110e7393-0f60-4b1a-b58f-62f216970c24/manager/0.log"
Sep 09 13:25:26.019221 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:26.019194    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647774d94-9jldd_f8b024cc-fea3-4dcc-b7d7-73b48fc94288/manager/0.log"
Sep 09 13:25:26.041770 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:26.041750    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647774d94-9jldd_f8b024cc-fea3-4dcc-b7d7-73b48fc94288/copy-manifests/0.log"
Sep 09 13:25:26.269025 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:26.268955    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cb5blq_5d270b03-da4f-49e1-932c-5bf1304ee88e/istio-proxy/0.log"
Sep 09 13:25:26.509226 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:26.509201    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hdh5c_3d2e3c5e-df61-41b0-b145-a1c4d245f844/discovery/0.log"
Sep 09 13:25:26.614963 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:26.614936    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rfh8m_5bfd0059-dde3-45f8-8570-612cfdd5a1fc/istio-proxy/0.log"
Sep 09 13:25:26.629338 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:26.629310    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rfh8m_5bfd0059-dde3-45f8-8570-612cfdd5a1fc/istio-proxy/1.log"
Sep 09 13:25:26.675406 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:26.675383    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gm2kv_7368e50b-ec89-4339-a32c-599551b9e8a2/payload-pre-processing/0.log"
Sep 09 13:25:27.659995 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:27.659969    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-798b96659f-nwsxc_bee787c5-ea08-40f4-a217-36256d1bfff7/manager/0.log"
Sep 09 13:25:28.213692 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:28.213666    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cb5blq_5d270b03-da4f-49e1-932c-5bf1304ee88e/istio-proxy/0.log"
Sep 09 13:25:28.646308 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:28.646283    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-hdh5c_3d2e3c5e-df61-41b0-b145-a1c4d245f844/discovery/0.log"
Sep 09 13:25:28.712795 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:28.712769    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rfh8m_5bfd0059-dde3-45f8-8570-612cfdd5a1fc/istio-proxy/0.log"
Sep 09 13:25:28.728018 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:28.727993    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-rfh8m_5bfd0059-dde3-45f8-8570-612cfdd5a1fc/istio-proxy/1.log"
Sep 09 13:25:28.767721 ip-10-0-132-160 kubenswrapper[2577]: I0909 13:25:28.767692    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-gm2kv_7368e50b-ec89-4339-a32c-599551b9e8a2/payload-pre-processing/0.log"