Sep 10 23:31:24.241829 ip-10-0-143-154 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 23:31:24.241841 ip-10-0-143-154 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 23:31:24.241850 ip-10-0-143-154 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 23:31:24.242186 ip-10-0-143-154 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 23:31:33.833465 ip-10-0-143-154 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 23:31:33.833478 ip-10-0-143-154 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a81fe89c1d1a44a18ad2c8a410f341a8 --
Sep 10 23:33:06.200292 ip-10-0-143-154 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 23:33:06.668475 ip-10-0-143-154 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 23:33:06.668475 ip-10-0-143-154 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 23:33:06.668475 ip-10-0-143-154 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 23:33:06.668475 ip-10-0-143-154 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 23:33:06.668475 ip-10-0-143-154 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 23:33:06.669391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.669308    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 23:33:06.674426 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674411    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:33:06.674426 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674426    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674430    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674433    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674436    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674439    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674442    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674445    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674447    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674450    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674453    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674457    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674466    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674481    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674484    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674487    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674490    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674493    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:33:06.674488 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674496    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674499    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674502    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674505    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674507    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674510    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674514    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674518    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674521    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674523    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674527    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674530    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674537    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674540    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674542    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674545    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674547    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674550    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674552    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674554    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:33:06.674921 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674561    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674564    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674567    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674569    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674572    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674574    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674578    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674580    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674583    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674586    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674589    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674591    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674594    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674597    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674600    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674603    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674605    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674608    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674611    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674613    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:33:06.675416 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674616    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674618    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674621    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674623    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674627    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674630    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674632    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674635    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674637    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674640    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674642    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674645    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674647    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674650    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674653    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674656    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674658    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674660    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674663    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674666    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:33:06.675902 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674668    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674671    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674673    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674676    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674678    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674681    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674684    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674686    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674689    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674693    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674696    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674699    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674702    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674704    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674707    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674710    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674713    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674716    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674718    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:33:06.676396 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674721    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.674723    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675416    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675422    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675425    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675429    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675433    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675437    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675440    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675443    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675446    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675449    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675451    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675454    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675456    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675459    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675462    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675464    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675467    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675470    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:33:06.676842 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675473    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675476    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675478    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675481    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675483    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675486    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675489    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675491    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675494    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675496    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675500    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675504    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675507    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675510    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675513    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675515    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675518    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675521    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675523    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675526    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:33:06.677374 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675529    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675532    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675534    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675537    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675540    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675543    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675547    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675549    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675552    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675554    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675557    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675560    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675563    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675565    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675568    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675571    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675573    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675576    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675578    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675581    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:33:06.677851 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675583    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675586    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675588    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675591    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675593    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675595    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675598    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675600    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675603    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675606    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675608    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675611    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675614    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675617    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675620    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675625    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675627    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675630    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675633    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:33:06.678355 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675635    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675638    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675642    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675645    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675649    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675652    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675654    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675657    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675660    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675662    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675665    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675667    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675670    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675673    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675675    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675678    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675680    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675683    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675685    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:33:06.678815 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675687    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675690    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.675693    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676070    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676075    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676078    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676081    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676084    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676087    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676092    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676094    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676097    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676100    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676102    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676105    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676108    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676111    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676114    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676116    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:33:06.679294 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676119    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676121    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676124    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676127    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676129    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676132    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676135    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676137    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676140    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676143    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676145    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676148    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676150    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676153    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676155    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676160    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676163    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676165    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676167    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:33:06.679759 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676171    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676174    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676178    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676182    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676184    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676187    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676189    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676192    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676195    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676197    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676200    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676202    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676205    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676207    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676210    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676212    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676215    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676217    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676220    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676222    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:33:06.680224 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676225    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676227    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676230    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676232    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676235    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676237    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676240    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676242    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676245    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676247    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676250    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676253    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676255    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676258    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676260    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676265    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676268    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676271    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676274    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676276    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:33:06.680699 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676279    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676282    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676284    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676287    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676289    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676292    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676294    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676297    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676299    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676302    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676304    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676307    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676309    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676312    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676314    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676318    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676322    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676325    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676328    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:33:06.681185 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676332    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676336    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676338    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676341    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.676344    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676414    2569 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676457    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676466    2569 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676474    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676479    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676483    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676488    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676492    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676496    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676499    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676502    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676505    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676508    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676511    2569 flags.go:64] FLAG: --cgroup-root=""
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676514    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676517    2569 flags.go:64] FLAG: --client-ca-file=""
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676520    2569 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676523    2569 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 23:33:06.681658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676528    2569 flags.go:64] FLAG: --cluster-domain=""
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676531    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676534    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676537    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676541    2569 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676544    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676547    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676551    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676554    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676557    2569 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676560    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676564    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676567    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676570    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676574    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676577    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676580    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676585    2569 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676588    2569 flags.go:64] FLAG: --enable-server="true"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676591    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676596    2569 flags.go:64] FLAG: --event-burst="100"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676600    2569 flags.go:64] FLAG: --event-qps="50"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676603    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676606    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 23:33:06.682236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676609    2569 flags.go:64] FLAG: --eviction-hard=""
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676612    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676615    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676618    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676622    2569 flags.go:64] FLAG: --eviction-soft=""
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676625    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676627    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676631    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676636    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676639    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676642    2569 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676644    2569 flags.go:64] FLAG: --feature-gates=""
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676648    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676651    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676655    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676658    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676661    2569 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676664    2569 flags.go:64] FLAG: --help="false"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676667    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-143-154.ec2.internal"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676670    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676673    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676676    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676679    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 23:33:06.682846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676683    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676686    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676690    2569 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676693    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676696    2569 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676699    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676703    2569 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676706    2569 flags.go:64] FLAG: --kube-reserved=""
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676709    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676712    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676715    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676718    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676721    2569 flags.go:64] FLAG: --lock-file=""
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676724    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676726    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676729    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676734    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676739    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676742    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676745    2569 flags.go:64] FLAG: --logging-format="text"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676748    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676751    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676754    2569 flags.go:64] FLAG: --manifest-url=""
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676757    2569 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676761    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 23:33:06.683693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676764    2569 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676768    2569 flags.go:64] FLAG: --max-pods="110"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676772    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676775    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676777    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676780    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676783    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676786    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676789    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676797    2569 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676800    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676803    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676806    2569 flags.go:64] FLAG: --pod-cidr=""
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676810    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676815    2569 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676818    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676821    2569 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676824    2569 flags.go:64] FLAG: --port="10250"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676827    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676830    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ecc1a605ed8acb75"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676833    2569 flags.go:64] FLAG: --qos-reserved=""
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676836    2569 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676839    2569 flags.go:64] FLAG: --register-node="true"
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676842    2569 flags.go:64] FLAG: --register-with-taints=""
Sep 10 23:33:06.684312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676847    2569 flags.go:64] FLAG: --registry-burst="10"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676850    2569 flags.go:64] FLAG: --registry-qps="5"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676853    2569 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676856    2569 flags.go:64] FLAG: --reserved-memory=""
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676860    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676863    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676866    2569 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676869    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676872    2569 flags.go:64] FLAG: --runonce="false"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676875    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676879    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676882    2569 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676885    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676888    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676891    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676894    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676897    2569 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676900    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676904    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676907    2569 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676909    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676913    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676916    2569 flags.go:64] FLAG: --system-cgroups=""
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676919    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676924    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 23:33:06.684897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676927    2569 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676930    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676946    2569 flags.go:64] FLAG: --tls-min-version=""
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676948    2569 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676951    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676954    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676957    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676961    2569 flags.go:64] FLAG: --v="2"
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676965    2569 flags.go:64] FLAG: --version="false"
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676969    2569 flags.go:64] FLAG: --vmodule=""
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676973    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.676976    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677098    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677103    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677106    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677109    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677111    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677114    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677117    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677120    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677123    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677126    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:33:06.685504 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677128    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677131    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677133    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677136    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677139    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677142    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677144    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677147    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677150    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677153    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677156    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677158    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677161    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677163    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677166    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677169    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677171    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677177    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677180    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677182    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:33:06.686103 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677185    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677188    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677190    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677193    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677197    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677200    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677204    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677207    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677209    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677212    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677215    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677218    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677221    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677225    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677228    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677231    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677234    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677237    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677239    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:33:06.686626 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677242    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677245    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677248    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677250    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677253    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677255    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677258    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677261    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677263    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677266    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677270    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677273    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677275    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677278    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677281    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677283    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677286    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677288    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677291    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677293    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:33:06.687105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677296    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677299    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677301    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677304    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677306    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677309    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677319    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677322    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677325    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677328    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677330    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677333    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677335    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677338    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677341    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677344    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677347    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677349    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677352    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677356    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:33:06.687585 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677359    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677362    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677366    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677369    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677372    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677374    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677377    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677380    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677383    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.677386    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.678074    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.685497    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.685511    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.685531    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.685540    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 23:33:06.688084 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685599    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685604    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685608    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685614    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685617    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685620    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685623    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685626    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685629    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685632    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685635    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685637    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685640    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685643    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685646    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685649    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685652    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685654    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685657    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:33:06.688445 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685660    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685663    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685665    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685668    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685671    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685673    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685676    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685679    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685681    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685684    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685687    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685690    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685693    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685695    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685698    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685701    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685704    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685706    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685709    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:33:06.688903 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685711    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685714    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685717    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685719    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685722    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685724    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685727    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685729    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685732    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685734    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685737    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685740    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685742    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685745    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685747    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685750    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685752    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685755    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685757    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685760    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:33:06.689377 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685762    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685765    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685767    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685770    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685773    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685776    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685778    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685781    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685784    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685786    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685789    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685792    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685794    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685797    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685799    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685802    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685804    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685808    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685812    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685815    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:33:06.690061 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685817    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685820    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685822    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685825    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685828    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685830    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685833    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685835    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685838    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685841    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685844    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685846    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685849    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685851    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685854    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685856    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685859    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685862    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685865    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685867    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:33:06.690534 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.685870    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.685874    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686003    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686008    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686013    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686017    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686020    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686023    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686026    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686029    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686032    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686034    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686037    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686040    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 23:33:06.691049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686042    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686045    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686047    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686050    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686053    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686055    2569 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686058    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686060    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686063    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686065    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686067    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686070    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686072    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686075    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686077    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686080    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686084    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686086    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686089    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686091    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 23:33:06.691385 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686094    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686097    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686099    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686102    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686105    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686107    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686110    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686113    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686116    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686118    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686121    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686123    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686126    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686129    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686131    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686134    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686137    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686139    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686142    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686144    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 23:33:06.691871 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686147    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686149    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686152    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686154    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686157    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686159    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686161    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686166    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686169    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686172    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686174    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686177    2569 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686180    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686183    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686185    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686188    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686191    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686195    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686197    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 23:33:06.692364 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686200    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686203    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686205    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686208    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686210    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686213    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686216    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686218    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686221    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686223    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686226    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686228    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686231    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686233    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686236    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686239    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686242    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686244    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686247    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686249    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 23:33:06.692813 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686253    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686256    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686260    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686263    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686266    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686270    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686273    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:06.686275    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.686280    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.686402    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.688830    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.689978    2569 server.go:1019] "Starting client certificate rotation"
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.690076    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 23:33:06.693395 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.690108    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 23:33:06.693900 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.692580    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 23:33:06.699498 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.699477    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 23:33:06.711248 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.711227    2569 log.go:25] "Validated CRI v1 runtime API"
Sep 10 23:33:06.721895 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.721871    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 23:33:06.771438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.771411    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qm8m9"
Sep 10 23:33:06.778873 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.778856    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qm8m9"
Sep 10 23:33:06.849653 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.849628    2569 log.go:25] "Validated CRI v1 image API"
Sep 10 23:33:06.850696 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.850653    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 23:33:06.857255 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.857229    2569 fs.go:135] Filesystem UUIDs: map[1e93ed3a-7189-43db-af1b-ca4127ec949d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cfb77f4d-5e89-46c1-9c36-d06361ef2d44:/dev/nvme0n1p3]
Sep 10 23:33:06.857337 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.857253    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 23:33:06.862803 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.862674    2569 manager.go:216] Machine: {Timestamp:2026-09-10 23:33:06.861218142 +0000 UTC m=+0.514938305 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100205 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2be806512266e0133ac9aff1b2335b SystemUUID:ec2be806-5122-66e0-133a-c9aff1b2335b BootID:a81fe89c-1d1a-44a1-8ad2-c8a410f341a8 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:88:f8:75:ca:e1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:88:f8:75:ca:e1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:58:83:40:b0:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 23:33:06.862803 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.862796    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 23:33:06.862926 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.862919    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 23:33:06.866376 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.866342    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 23:33:06.866518 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.866378    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-154.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 23:33:06.867249 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.867237    2569 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 23:33:06.867282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.867251    2569 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 23:33:06.867282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.867259    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 23:33:06.867282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.867274    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 23:33:06.870562 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.870548    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 23:33:06.877416 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.877401    2569 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 23:33:06.877550 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.877540    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 23:33:06.877580 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.877571    2569 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 23:33:06.877615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.877585    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 23:33:06.877615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.877603    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 23:33:06.877615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.877613    2569 kubelet.go:398] "Adding apiserver pod source"
Sep 10 23:33:06.877691 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.877626    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 23:33:06.878126 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.878100    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 23:33:06.878126 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.878125    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 23:33:06.880725 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.880703    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 23:33:06.883975 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.883961    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 23:33:06.884041 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.883991    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 23:33:06.886072 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886059    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 23:33:06.886115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886081    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 23:33:06.886115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886092    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 23:33:06.886115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886101    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 23:33:06.886115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886114    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 23:33:06.886220 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886123    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 23:33:06.886220 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886142    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 23:33:06.886220 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886151    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 23:33:06.886220 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886169    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 23:33:06.886220 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886185    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 23:33:06.886220 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886197    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 23:33:06.886220 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.886206    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 23:33:06.887188 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.887171    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 23:33:06.887188 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.887191    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 23:33:06.891684 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.891640    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:33:06.894337 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.894317    2569 server.go:1268] "Started kubelet"
Sep 10 23:33:06.894434 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.894408    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 23:33:06.894505 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.894460    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 23:33:06.894551 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.894528    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 23:33:06.894640 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.894622    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:33:06.895281 ip-10-0-143-154 systemd[1]: Started Kubernetes Kubelet.
Sep 10 23:33:06.896144 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.896122    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 23:33:06.896684 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.896668    2569 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 23:33:06.896765 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.896722    2569 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-154.ec2.internal" not found
Sep 10 23:33:06.901129 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:06.901064    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 23:33:06.904538 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.904522    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 23:33:06.904538 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.904535    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 23:33:06.904663 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:06.904599    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-154.ec2.internal\" not found"
Sep 10 23:33:06.905441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905256    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 23:33:06.905441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905274    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 23:33:06.905441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905356    2569 factory.go:55] Registering systemd factory
Sep 10 23:33:06.905441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905371    2569 factory.go:223] Registration of the systemd container factory successfully
Sep 10 23:33:06.905441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905408    2569 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 23:33:06.905441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905417    2569 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 23:33:06.905789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905444    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 23:33:06.905789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905654    2569 factory.go:153] Registering CRI-O factory
Sep 10 23:33:06.905789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905669    2569 factory.go:223] Registration of the crio container factory successfully
Sep 10 23:33:06.905789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905726    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 23:33:06.905789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905746    2569 factory.go:103] Registering Raw factory
Sep 10 23:33:06.905789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.905760    2569 manager.go:1196] Started watching for new ooms in manager
Sep 10 23:33:06.906849 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.906831    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:33:06.906980 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.906950    2569 manager.go:349] Starting recovery of all containers
Sep 10 23:33:06.910261 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:06.910237    2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-154.ec2.internal\" not found" node="ip-10-0-143-154.ec2.internal"
Sep 10 23:33:06.913707 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.913688    2569 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-154.ec2.internal" not found
Sep 10 23:33:06.917711 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.917691    2569 manager.go:354] Recovery completed
Sep 10 23:33:06.923345 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.923331    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:33:06.926576 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.926558    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:33:06.926636 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.926591    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:33:06.926636 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.926602    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:33:06.927177 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.927163    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 23:33:06.927177 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.927175    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 23:33:06.927268 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.927193    2569 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 23:33:06.927268 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.927233    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:33:06.928045 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.928031    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:33:06.928093 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.928056    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:33:06.928093 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.928067    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:33:06.929391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.929371    2569 policy_none.go:49] "None policy: Start"
Sep 10 23:33:06.929391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.929389    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 23:33:06.929495 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.929402    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 23:33:06.931375 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.931361    2569 policy_none.go:47] "Start"
Sep 10 23:33:06.967534 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.967519    2569 manager.go:337] "Starting Device Plugin manager"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:06.967562    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.967575    2569 server.go:87] "Starting device plugin registration server"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.967809    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.967819    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.967949    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.968038    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.968047    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:06.968500    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:06.968541    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-154.ec2.internal\" not found"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.970417    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.971308    2569 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-154.ec2.internal" not found
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.971562    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.971610    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.971635    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:06.971680    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 23:33:06.976276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:06.973798    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:33:07.068714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.068674    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:33:07.069775 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.069758    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:33:07.069850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.069790    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:33:07.069850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.069800    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:33:07.069850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.069825    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.071942 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.071914    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"]
Sep 10 23:33:07.071999 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.071989    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:33:07.073435 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.073406    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:33:07.073435 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.073434    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:33:07.073573 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.073444    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:33:07.075747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.075734    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:33:07.075876 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.075862    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.075945 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.075897    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 23:33:07.076440 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.076426    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:33:07.076502 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.076473    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:33:07.076502 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.076484    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:33:07.076599 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.076431    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 23:33:07.076599 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.076527    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 23:33:07.076599 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.076542    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeHasSufficientPID"
Sep 10 23:33:07.077433 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.077418    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.077481 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:07.077442    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-154.ec2.internal\": node \"ip-10-0-143-154.ec2.internal\" not found"
Sep 10 23:33:07.078767 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.078754    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.098477 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.098460    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.102872 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.102859    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.104910 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.104893    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.106732 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.106716    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45284353ad5e38e301b33a8146d7b581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"45284353ad5e38e301b33a8146d7b581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.106806 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.106744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed43c03ec92d5b25a955c89d154a23b6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-154.ec2.internal\" (UID: \"ed43c03ec92d5b25a955c89d154a23b6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.106806 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.106767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45284353ad5e38e301b33a8146d7b581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"45284353ad5e38e301b33a8146d7b581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.197046 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.196961    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 23:33:07.200103 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.200087    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 23:33:07.203597 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.203583    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 23:33:07.203647 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:07.203623    2569 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.203647 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.203635    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.207266 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.207252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45284353ad5e38e301b33a8146d7b581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"45284353ad5e38e301b33a8146d7b581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.207310 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.207273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45284353ad5e38e301b33a8146d7b581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"45284353ad5e38e301b33a8146d7b581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.207310 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.207288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed43c03ec92d5b25a955c89d154a23b6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-154.ec2.internal\" (UID: \"ed43c03ec92d5b25a955c89d154a23b6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.207374 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.207338    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ed43c03ec92d5b25a955c89d154a23b6-config\") pod \"kube-apiserver-proxy-ip-10-0-143-154.ec2.internal\" (UID: \"ed43c03ec92d5b25a955c89d154a23b6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.207374 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.207352    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45284353ad5e38e301b33a8146d7b581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"45284353ad5e38e301b33a8146d7b581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.207374 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.207358    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45284353ad5e38e301b33a8146d7b581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal\" (UID: \"45284353ad5e38e301b33a8146d7b581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.214922 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.214905    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 23:33:07.215018 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:07.214968    2569 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-154.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.502132 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.501888    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.503989 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.503973    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal"
Sep 10 23:33:07.689361 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.689324    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 23:33:07.689812 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.689508    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 23:33:07.689812 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.689512    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 23:33:07.689812 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.689508    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 23:33:07.853707 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.853604    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 23:28:06 +0000 UTC" deadline="2028-06-13 17:53:53.466307448 +0000 UTC"
Sep 10 23:33:07.853707 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.853651    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15402h20m45.612659595s"
Sep 10 23:33:07.878784 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.878755    2569 apiserver.go:52] "Watching apiserver"
Sep 10 23:33:07.887072 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.887044    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 23:33:07.887386 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.887366    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-9qt4w","openshift-multus/multus-additional-cni-plugins-t6th9","openshift-network-diagnostics/network-check-target-vbczg","openshift-ovn-kubernetes/ovnkube-node-zxbz2","kube-system/global-pull-secret-syncer-lgb42","kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal","openshift-cluster-node-tuning-operator/tuned-82p42","openshift-dns/node-resolver-4fj9t","openshift-image-registry/node-ca-7vxkh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal","openshift-multus/network-metrics-daemon-mh7bp","openshift-network-operator/iptables-alerter-wz7cc","kube-system/konnectivity-agent-l297b","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"]
Sep 10 23:33:07.889734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.889718    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.891670 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.891645    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 23:33:07.891670 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.891660    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 23:33:07.891820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.891650    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 23:33:07.891820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.891725    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:07.891820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.891650    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jrgl2\""
Sep 10 23:33:07.891820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.891758    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 23:33:07.893367 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.893350    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n6ngn\""
Sep 10 23:33:07.893436 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.893383    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 23:33:07.893469 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.893383    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 23:33:07.893693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.893679    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:07.893760 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:07.893722    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:07.895743 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.895729    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.897401 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.897385    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 23:33:07.897475 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.897454    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 23:33:07.897513 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.897494    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 23:33:07.897559 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.897494    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-w4lw6\""
Sep 10 23:33:07.897559 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.897552    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 23:33:07.897694 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.897675    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 23:33:07.897694 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.897679    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 23:33:07.897858 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.897844    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:07.897925 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:07.897905    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:07.900034 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.900015    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.901651 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.901638    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:33:07.901738 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.901715    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 23:33:07.901738 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.901733    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cz6s9\""
Sep 10 23:33:07.902245 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.902227    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:07.903661 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.903647    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 23:33:07.903755 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.903739    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 23:33:07.903802 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.903738    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8g5v7\""
Sep 10 23:33:07.904245 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.904231    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:07.904715 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.904699    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 23:33:07.905704 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.905686    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 23:33:07.905785 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.905715    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 23:33:07.905965 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.905948    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 23:33:07.906036 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.906021    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-94hwz\""
Sep 10 23:33:07.906623 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.906357    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:07.906623 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:07.906407    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:07.908407 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.908393    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:07.910340 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.910323    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 23:33:07.910534 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.910520    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:33:07.910996 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.910980    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 23:33:07.911274 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.911256    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m8xmb\""
Sep 10 23:33:07.911736 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.911709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-cnibin\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:07.911827 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.911746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-cni-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.911827 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.911802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-systemd\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.911827 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.911822    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:07.912002 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.911842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.912002 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.911886    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-log-socket\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.912087 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysctl-d\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.912087 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c29833e6-791a-469b-931e-c40cda4afd2a-tmp\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.912087 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912072    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-kubelet\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.912253 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912114    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpq9p\" (UniqueName: \"kubernetes.io/projected/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-kube-api-access-vpq9p\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.912253 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912170    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-systemd-units\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.912253 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912202    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-system-cni-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.912559 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-lib-modules\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.912613 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912589    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c29833e6-791a-469b-931e-c40cda4afd2a-etc-tuned\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.912658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rntnw\" (UniqueName: \"kubernetes.io/projected/256ec37b-d6e3-436b-8fba-c5046b103926-kube-api-access-rntnw\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:07.912658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912642    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-slash\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.912767 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysconfig\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.912767 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f24f3d3d-137d-4393-b095-e651175daf4c-tmp-dir\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:07.912767 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912711    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.912767 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ktb\" (UniqueName: \"kubernetes.io/projected/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-kube-api-access-27ktb\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.912767 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912760    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-var-lib-kubelet\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.913021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912793    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-run-netns\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912824    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-cni-bin\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-node-log\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912881    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c5e1758-ac02-4c25-8c56-7f229ac909b7-dbus\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:07.913021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-netns\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.913021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.912976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysctl-conf\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.913021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-sys\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913041    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-system-cni-dir\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913079    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913098    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-cni-netd\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-cni-binary-copy\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913133    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-host\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913154    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-k8s-cni-cncf-io\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwkvx\" (UniqueName: \"kubernetes.io/projected/c29833e6-791a-469b-931e-c40cda4afd2a-kube-api-access-cwkvx\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913194    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e74baf0-dfa1-45b8-87ba-8fb556a20495-serviceca\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913212    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-etc-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913225    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-cni-multus\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-daemon-config\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-modprobe-d\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.913297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovnkube-config\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-systemd\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-var-lib-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913399    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c5e1758-ac02-4c25-8c56-7f229ac909b7-kubelet-config\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913405    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r64n4\""
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913419    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f24f3d3d-137d-4393-b095-e651175daf4c-hosts-file\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913439    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e74baf0-dfa1-45b8-87ba-8fb556a20495-host\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913471    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-env-overrides\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdh88\" (UniqueName: \"kubernetes.io/projected/f24f3d3d-137d-4393-b095-e651175daf4c-kube-api-access-sdh88\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913509    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tppvg\" (UniqueName: \"kubernetes.io/projected/1e74baf0-dfa1-45b8-87ba-8fb556a20495-kube-api-access-tppvg\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913531    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913584    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovnkube-script-lib\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913604    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913631    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6x58\" (UniqueName: \"kubernetes.io/projected/5dd6ce58-5b02-4e04-9463-38ccc4773b66-kube-api-access-s6x58\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913651    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-hostroot\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913671    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-multus-certs\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.913879 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913683    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913695    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-kubernetes\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913694    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913713    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-cni-bin\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913726    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-etc-kubernetes\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913741    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovn-node-metrics-cert\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-kubelet\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913774    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-ovn\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913792    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-cnibin\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-socket-dir-parent\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913834    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-conf-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-run\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913875    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-cni-binary-copy\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:07.914357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.913901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:07.914773 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.914610    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:07.916207 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.916174    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 23:33:07.916207 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.916185    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 23:33:07.916321 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.916308    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vrz9f\""
Sep 10 23:33:07.916321 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.916312    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 23:33:07.918462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.918446    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 23:33:07.933119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.933007    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-94jxt"
Sep 10 23:33:07.942260 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:07.942240    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-94jxt"
Sep 10 23:33:08.006443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.006419    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 23:33:08.014196 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:08.014196 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-run\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.014366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014216    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-cni-binary-copy\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.014366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.014366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014245    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.014366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-iptables-alerter-script\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.014366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014295    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7pq9\" (UniqueName: \"kubernetes.io/projected/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-kube-api-access-j7pq9\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.014366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-registration-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.014366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-cnibin\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.014366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014349    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-cni-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014373    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-systemd\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014400    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014412    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-cnibin\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-log-socket\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-run\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014452    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysctl-d\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014460    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-systemd\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014479    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c29833e6-791a-469b-931e-c40cda4afd2a-tmp\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014479    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014510    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-log-socket\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014413    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-etc-selinux\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014565    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-kubelet\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014556    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-cni-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpq9p\" (UniqueName: \"kubernetes.io/projected/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-kube-api-access-vpq9p\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-kubelet\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014577    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysctl-d\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.014726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-systemd-units\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-system-cni-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014689    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-lib-modules\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014717    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c29833e6-791a-469b-931e-c40cda4afd2a-etc-tuned\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014763    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-system-cni-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014768    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014770    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014812    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014716    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-systemd-units\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014805    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rntnw\" (UniqueName: \"kubernetes.io/projected/256ec37b-d6e3-436b-8fba-c5046b103926-kube-api-access-rntnw\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014947    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-lib-modules\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014963    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-slash\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014992    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-slash\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-cni-binary-copy\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.014990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysconfig\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysconfig\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015044    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f24f3d3d-137d-4393-b095-e651175daf4c-tmp-dir\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:08.015565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015088    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-socket-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015122    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-device-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015147    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015180    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27ktb\" (UniqueName: \"kubernetes.io/projected/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-kube-api-access-27ktb\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-var-lib-kubelet\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015228    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-run-ovn-kubernetes\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-run-netns\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015274    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f24f3d3d-137d-4393-b095-e651175daf4c-tmp-dir\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015281    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-cni-bin\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-node-log\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015326    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-cni-bin\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015332    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-node-log\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015327    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-run-netns\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015345    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-var-lib-kubelet\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c5e1758-ac02-4c25-8c56-7f229ac909b7-dbus\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-netns\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015423    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysctl-conf\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015438    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4c5e1758-ac02-4c25-8c56-7f229ac909b7-dbus\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:08.016250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015446    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-sys\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015426    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-netns\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015491    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-system-cni-dir\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015516    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-sys\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015546    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-sysctl-conf\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015467    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/256ec37b-d6e3-436b-8fba-c5046b103926-system-cni-dir\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015580    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-cni-netd\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-cni-binary-copy\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015642    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-cni-netd\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-host\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015702    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-host-slash\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015723    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-sys-fs\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015758    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-k8s-cni-cncf-io\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015779    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-host\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015797    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwkvx\" (UniqueName: \"kubernetes.io/projected/c29833e6-791a-469b-931e-c40cda4afd2a-kube-api-access-cwkvx\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015803    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-k8s-cni-cncf-io\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.017082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015862    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e74baf0-dfa1-45b8-87ba-8fb556a20495-serviceca\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015924    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29d288de-88fe-491c-a6fd-4edcd4cd5541-agent-certs\") pod \"konnectivity-agent-l297b\" (UID: \"29d288de-88fe-491c-a6fd-4edcd4cd5541\") " pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cswdd\" (UniqueName: \"kubernetes.io/projected/b7412ae6-9d63-439a-b129-3036ca7ac110-kube-api-access-cswdd\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.015995    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-etc-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-cni-multus\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016043    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/256ec37b-d6e3-436b-8fba-c5046b103926-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-daemon-config\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-modprobe-d\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016139    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovnkube-config\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-systemd\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-cni-binary-copy\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016244    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-systemd\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-var-lib-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c5e1758-ac02-4c25-8c56-7f229ac909b7-kubelet-config\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016353    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f24f3d3d-137d-4393-b095-e651175daf4c-hosts-file\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016390    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e74baf0-dfa1-45b8-87ba-8fb556a20495-host\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016430    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-modprobe-d\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.018391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016458    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f24f3d3d-137d-4393-b095-e651175daf4c-hosts-file\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016486    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4c5e1758-ac02-4c25-8c56-7f229ac909b7-kubelet-config\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016498    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-var-lib-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016513    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-daemon-config\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016538    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-env-overrides\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016564    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-etc-openvswitch\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016586    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-cni-multus\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016592    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdh88\" (UniqueName: \"kubernetes.io/projected/f24f3d3d-137d-4393-b095-e651175daf4c-kube-api-access-sdh88\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016632    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e74baf0-dfa1-45b8-87ba-8fb556a20495-host\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016625    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tppvg\" (UniqueName: \"kubernetes.io/projected/1e74baf0-dfa1-45b8-87ba-8fb556a20495-kube-api-access-tppvg\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016746    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e74baf0-dfa1-45b8-87ba-8fb556a20495-serviceca\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016766    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovnkube-config\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovnkube-script-lib\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016949    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016888    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.016989    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6x58\" (UniqueName: \"kubernetes.io/projected/5dd6ce58-5b02-4e04-9463-38ccc4773b66-kube-api-access-s6x58\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:08.019102 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.017038    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-env-overrides\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-hostroot\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.017099    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret podName:4c5e1758-ac02-4c25-8c56-7f229ac909b7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:08.517076604 +0000 UTC m=+2.170796770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret") pod "global-pull-secret-syncer-lgb42" (UID: "4c5e1758-ac02-4c25-8c56-7f229ac909b7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-multus-certs\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.017122    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-kubernetes\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.017178    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs podName:5dd6ce58-5b02-4e04-9463-38ccc4773b66 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:08.517164055 +0000 UTC m=+2.170884219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs") pod "network-metrics-daemon-mh7bp" (UID: "5dd6ce58-5b02-4e04-9463-38ccc4773b66") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-cni-bin\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017207    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c29833e6-791a-469b-931e-c40cda4afd2a-etc-kubernetes\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017207    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-hostroot\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-etc-kubernetes\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-run-multus-certs\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29d288de-88fe-491c-a6fd-4edcd4cd5541-konnectivity-ca\") pod \"konnectivity-agent-l297b\" (UID: \"29d288de-88fe-491c-a6fd-4edcd4cd5541\") " pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017255    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-host-var-lib-cni-bin\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017275    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovnkube-script-lib\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovn-node-metrics-cert\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.019986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017284    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-etc-kubernetes\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017325    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-kubelet\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017349    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-ovn\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017400    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-cnibin\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-run-ovn\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017423    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-socket-dir-parent\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-socket-dir-parent\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017475    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-host-kubelet\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017508    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-conf-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017670    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-cnibin\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.017683    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-multus-conf-dir\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.018898    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c29833e6-791a-469b-931e-c40cda4afd2a-etc-tuned\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.019157    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c29833e6-791a-469b-931e-c40cda4afd2a-tmp\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.020740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.019914    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-ovn-node-metrics-cert\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.023698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.023678    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:33:08.023698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.023702    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:33:08.023698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.023715    2569 projected.go:196] Error preparing data for projected volume kube-api-access-cmf4d for pod openshift-network-diagnostics/network-check-target-vbczg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:08.023989 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.023793    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d podName:57fdc049-4633-4a9d-a899-d13dae2f0b81 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:08.5237742 +0000 UTC m=+2.177494352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cmf4d" (UniqueName: "kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d") pod "network-check-target-vbczg" (UID: "57fdc049-4633-4a9d-a899-d13dae2f0b81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:08.027256 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.027041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpq9p\" (UniqueName: \"kubernetes.io/projected/85a41d6c-95e7-4a39-99e6-6b5ff81acfc3-kube-api-access-vpq9p\") pod \"multus-9qt4w\" (UID: \"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3\") " pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.027561 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.027540    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdh88\" (UniqueName: \"kubernetes.io/projected/f24f3d3d-137d-4393-b095-e651175daf4c-kube-api-access-sdh88\") pod \"node-resolver-4fj9t\" (UID: \"f24f3d3d-137d-4393-b095-e651175daf4c\") " pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:08.028095 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.028072    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rntnw\" (UniqueName: \"kubernetes.io/projected/256ec37b-d6e3-436b-8fba-c5046b103926-kube-api-access-rntnw\") pod \"multus-additional-cni-plugins-t6th9\" (UID: \"256ec37b-d6e3-436b-8fba-c5046b103926\") " pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.029215 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.029192    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ktb\" (UniqueName: \"kubernetes.io/projected/2b0cbfcf-07ff-4607-939b-7eff0b21acf9-kube-api-access-27ktb\") pod \"ovnkube-node-zxbz2\" (UID: \"2b0cbfcf-07ff-4607-939b-7eff0b21acf9\") " pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.029456 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.029431    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tppvg\" (UniqueName: \"kubernetes.io/projected/1e74baf0-dfa1-45b8-87ba-8fb556a20495-kube-api-access-tppvg\") pod \"node-ca-7vxkh\" (UID: \"1e74baf0-dfa1-45b8-87ba-8fb556a20495\") " pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:08.029664 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.029649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6x58\" (UniqueName: \"kubernetes.io/projected/5dd6ce58-5b02-4e04-9463-38ccc4773b66-kube-api-access-s6x58\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:08.029903 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.029883    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwkvx\" (UniqueName: \"kubernetes.io/projected/c29833e6-791a-469b-931e-c40cda4afd2a-kube-api-access-cwkvx\") pod \"tuned-82p42\" (UID: \"c29833e6-791a-469b-931e-c40cda4afd2a\") " pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.118433 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118409    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-host-slash\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.118532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-sys-fs\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118455    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29d288de-88fe-491c-a6fd-4edcd4cd5541-agent-certs\") pod \"konnectivity-agent-l297b\" (UID: \"29d288de-88fe-491c-a6fd-4edcd4cd5541\") " pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:08.118532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118471    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cswdd\" (UniqueName: \"kubernetes.io/projected/b7412ae6-9d63-439a-b129-3036ca7ac110-kube-api-access-cswdd\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118524    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-host-slash\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.118633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118530    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-sys-fs\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118532    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29d288de-88fe-491c-a6fd-4edcd4cd5541-konnectivity-ca\") pod \"konnectivity-agent-l297b\" (UID: \"29d288de-88fe-491c-a6fd-4edcd4cd5541\") " pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:08.118633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-iptables-alerter-script\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.118781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118637    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7pq9\" (UniqueName: \"kubernetes.io/projected/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-kube-api-access-j7pq9\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.118781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118665    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-registration-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118692    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-etc-selinux\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118762    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-registration-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-etc-selinux\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-socket-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-device-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118895    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-device-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.118984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.118970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7412ae6-9d63-439a-b129-3036ca7ac110-socket-dir\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.119276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.119016    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29d288de-88fe-491c-a6fd-4edcd4cd5541-konnectivity-ca\") pod \"konnectivity-agent-l297b\" (UID: \"29d288de-88fe-491c-a6fd-4edcd4cd5541\") " pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:08.119276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.119209    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-iptables-alerter-script\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.120576 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.120558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29d288de-88fe-491c-a6fd-4edcd4cd5541-agent-certs\") pod \"konnectivity-agent-l297b\" (UID: \"29d288de-88fe-491c-a6fd-4edcd4cd5541\") " pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:08.125556 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.125531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7pq9\" (UniqueName: \"kubernetes.io/projected/d0dc0926-af13-418a-9c7d-b82d0afa5bd6-kube-api-access-j7pq9\") pod \"iptables-alerter-wz7cc\" (UID: \"d0dc0926-af13-418a-9c7d-b82d0afa5bd6\") " pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.125806 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.125793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cswdd\" (UniqueName: \"kubernetes.io/projected/b7412ae6-9d63-439a-b129-3036ca7ac110-kube-api-access-cswdd\") pod \"aws-ebs-csi-driver-node-5ch62\" (UID: \"b7412ae6-9d63-439a-b129-3036ca7ac110\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.128073 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.128047    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45284353ad5e38e301b33a8146d7b581.slice/crio-123f8d24e00c053c121b70f3639a3162b6aeebafb73f30ffb9a3855d56c3c248 WatchSource:0}: Error finding container 123f8d24e00c053c121b70f3639a3162b6aeebafb73f30ffb9a3855d56c3c248: Status 404 returned error can't find the container with id 123f8d24e00c053c121b70f3639a3162b6aeebafb73f30ffb9a3855d56c3c248
Sep 10 23:33:08.128237 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.128221    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded43c03ec92d5b25a955c89d154a23b6.slice/crio-f35301990e865b17abadb8815cf9fe2c3d2de35628fb5ab6af2ddc06e877a3a4 WatchSource:0}: Error finding container f35301990e865b17abadb8815cf9fe2c3d2de35628fb5ab6af2ddc06e877a3a4: Status 404 returned error can't find the container with id f35301990e865b17abadb8815cf9fe2c3d2de35628fb5ab6af2ddc06e877a3a4
Sep 10 23:33:08.132523 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.132509    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:33:08.213680 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.213634    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9qt4w"
Sep 10 23:33:08.222775 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.222750    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85a41d6c_95e7_4a39_99e6_6b5ff81acfc3.slice/crio-3d93008bea69b980510f38c919f5a05aab6efea97ba794d82daa6d23e8f85056 WatchSource:0}: Error finding container 3d93008bea69b980510f38c919f5a05aab6efea97ba794d82daa6d23e8f85056: Status 404 returned error can't find the container with id 3d93008bea69b980510f38c919f5a05aab6efea97ba794d82daa6d23e8f85056
Sep 10 23:33:08.234985 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.234968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t6th9"
Sep 10 23:33:08.242133 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.242104    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod256ec37b_d6e3_436b_8fba_c5046b103926.slice/crio-fc1537c0c9f1e0d894e79a0ea21917db3dc0becf3f29d4bf55a2ba815adc3381 WatchSource:0}: Error finding container fc1537c0c9f1e0d894e79a0ea21917db3dc0becf3f29d4bf55a2ba815adc3381: Status 404 returned error can't find the container with id fc1537c0c9f1e0d894e79a0ea21917db3dc0becf3f29d4bf55a2ba815adc3381
Sep 10 23:33:08.250447 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.250429    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:08.257453 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.257433    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b0cbfcf_07ff_4607_939b_7eff0b21acf9.slice/crio-02721651acb021e053ff50e6bd23de16d66907622d88b4156338891ac33d21fd WatchSource:0}: Error finding container 02721651acb021e053ff50e6bd23de16d66907622d88b4156338891ac33d21fd: Status 404 returned error can't find the container with id 02721651acb021e053ff50e6bd23de16d66907622d88b4156338891ac33d21fd
Sep 10 23:33:08.266838 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.266822    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-82p42"
Sep 10 23:33:08.273381 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.273364    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4fj9t"
Sep 10 23:33:08.274099 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.274082    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29833e6_791a_469b_931e_c40cda4afd2a.slice/crio-f4ed86f92a6e74ab9d0f12615c83478ea526c4f9cc3f4d635c3a07064f3f717f WatchSource:0}: Error finding container f4ed86f92a6e74ab9d0f12615c83478ea526c4f9cc3f4d635c3a07064f3f717f: Status 404 returned error can't find the container with id f4ed86f92a6e74ab9d0f12615c83478ea526c4f9cc3f4d635c3a07064f3f717f
Sep 10 23:33:08.280562 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.280537    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24f3d3d_137d_4393_b095_e651175daf4c.slice/crio-faf59a159b4bd9bffd0f28769ea3ed0b219b8ea4ad02bc6a785260a29147496a WatchSource:0}: Error finding container faf59a159b4bd9bffd0f28769ea3ed0b219b8ea4ad02bc6a785260a29147496a: Status 404 returned error can't find the container with id faf59a159b4bd9bffd0f28769ea3ed0b219b8ea4ad02bc6a785260a29147496a
Sep 10 23:33:08.295092 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.295066    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vxkh"
Sep 10 23:33:08.302912 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.302891    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e74baf0_dfa1_45b8_87ba_8fb556a20495.slice/crio-e270b6dfc0a16906a18880d014fb191689f0a8a02b56e8bd48489ba251f6885f WatchSource:0}: Error finding container e270b6dfc0a16906a18880d014fb191689f0a8a02b56e8bd48489ba251f6885f: Status 404 returned error can't find the container with id e270b6dfc0a16906a18880d014fb191689f0a8a02b56e8bd48489ba251f6885f
Sep 10 23:33:08.333052 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.333028    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wz7cc"
Sep 10 23:33:08.341514 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.341489    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0dc0926_af13_418a_9c7d_b82d0afa5bd6.slice/crio-53fae09cab89ae43829aa6fbf1409e9fb1d0975ddfed5e9d84ece3f8a6d2d962 WatchSource:0}: Error finding container 53fae09cab89ae43829aa6fbf1409e9fb1d0975ddfed5e9d84ece3f8a6d2d962: Status 404 returned error can't find the container with id 53fae09cab89ae43829aa6fbf1409e9fb1d0975ddfed5e9d84ece3f8a6d2d962
Sep 10 23:33:08.347568 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.347552    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:08.351378 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.351358    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62"
Sep 10 23:33:08.355572 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:08.355551    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29d288de_88fe_491c_a6fd_4edcd4cd5541.slice/crio-87ba3290ef749fe3daf644541f7b6a15fbc1b106d739c06aa0c3d823f15aadce WatchSource:0}: Error finding container 87ba3290ef749fe3daf644541f7b6a15fbc1b106d739c06aa0c3d823f15aadce: Status 404 returned error can't find the container with id 87ba3290ef749fe3daf644541f7b6a15fbc1b106d739c06aa0c3d823f15aadce
Sep 10 23:33:08.521566 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.521486    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:08.521566 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.521531    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:08.521780 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.521653    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:08.521780 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.521660    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:08.521780 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.521704    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs podName:5dd6ce58-5b02-4e04-9463-38ccc4773b66 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:09.521691519 +0000 UTC m=+3.175411671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs") pod "network-metrics-daemon-mh7bp" (UID: "5dd6ce58-5b02-4e04-9463-38ccc4773b66") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:08.521780 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.521735    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret podName:4c5e1758-ac02-4c25-8c56-7f229ac909b7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:09.521715831 +0000 UTC m=+3.175435982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret") pod "global-pull-secret-syncer-lgb42" (UID: "4c5e1758-ac02-4c25-8c56-7f229ac909b7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:08.621952 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.621903    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:08.622142 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.622121    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:33:08.622221 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.622144    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:33:08.622221 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.622158    2569 projected.go:196] Error preparing data for projected volume kube-api-access-cmf4d for pod openshift-network-diagnostics/network-check-target-vbczg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:08.622324 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.622222    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d podName:57fdc049-4633-4a9d-a899-d13dae2f0b81 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:09.622202266 +0000 UTC m=+3.275922423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cmf4d" (UniqueName: "kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d") pod "network-check-target-vbczg" (UID: "57fdc049-4633-4a9d-a899-d13dae2f0b81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:08.764134 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.764101    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:33:08.837294 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.837008    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:33:08.944566 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.944512    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 23:28:07 +0000 UTC" deadline="2028-04-28 19:24:22.694288549 +0000 UTC"
Sep 10 23:33:08.944566 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.944565    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14299h51m13.749728371s"
Sep 10 23:33:08.973274 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.972608    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:08.973274 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:08.972758    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:08.989380 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.989323    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9qt4w" event={"ID":"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3","Type":"ContainerStarted","Data":"3d93008bea69b980510f38c919f5a05aab6efea97ba794d82daa6d23e8f85056"}
Sep 10 23:33:08.994815 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:08.994781    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wz7cc" event={"ID":"d0dc0926-af13-418a-9c7d-b82d0afa5bd6","Type":"ContainerStarted","Data":"53fae09cab89ae43829aa6fbf1409e9fb1d0975ddfed5e9d84ece3f8a6d2d962"}
Sep 10 23:33:09.010651 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.010607    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4fj9t" event={"ID":"f24f3d3d-137d-4393-b095-e651175daf4c","Type":"ContainerStarted","Data":"faf59a159b4bd9bffd0f28769ea3ed0b219b8ea4ad02bc6a785260a29147496a"}
Sep 10 23:33:09.035203 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.035104    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-82p42" event={"ID":"c29833e6-791a-469b-931e-c40cda4afd2a","Type":"ContainerStarted","Data":"f4ed86f92a6e74ab9d0f12615c83478ea526c4f9cc3f4d635c3a07064f3f717f"}
Sep 10 23:33:09.040412 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.040327    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6th9" event={"ID":"256ec37b-d6e3-436b-8fba-c5046b103926","Type":"ContainerStarted","Data":"fc1537c0c9f1e0d894e79a0ea21917db3dc0becf3f29d4bf55a2ba815adc3381"}
Sep 10 23:33:09.047404 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.047360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" event={"ID":"ed43c03ec92d5b25a955c89d154a23b6","Type":"ContainerStarted","Data":"f35301990e865b17abadb8815cf9fe2c3d2de35628fb5ab6af2ddc06e877a3a4"}
Sep 10 23:33:09.055651 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.055625    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 23:33:09.064198 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.064116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" event={"ID":"45284353ad5e38e301b33a8146d7b581","Type":"ContainerStarted","Data":"123f8d24e00c053c121b70f3639a3162b6aeebafb73f30ffb9a3855d56c3c248"}
Sep 10 23:33:09.071505 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.071469    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62" event={"ID":"b7412ae6-9d63-439a-b129-3036ca7ac110","Type":"ContainerStarted","Data":"a23d2b1c9a82224c4f6567d7cef4268a7240fb9c6dd3a32ad6a5132671ab45e1"}
Sep 10 23:33:09.077858 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.077824    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l297b" event={"ID":"29d288de-88fe-491c-a6fd-4edcd4cd5541","Type":"ContainerStarted","Data":"87ba3290ef749fe3daf644541f7b6a15fbc1b106d739c06aa0c3d823f15aadce"}
Sep 10 23:33:09.091707 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.091628    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vxkh" event={"ID":"1e74baf0-dfa1-45b8-87ba-8fb556a20495","Type":"ContainerStarted","Data":"e270b6dfc0a16906a18880d014fb191689f0a8a02b56e8bd48489ba251f6885f"}
Sep 10 23:33:09.109884 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.109849    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"02721651acb021e053ff50e6bd23de16d66907622d88b4156338891ac33d21fd"}
Sep 10 23:33:09.531112 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.531072    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:09.531348 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.531123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:09.531348 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.531233    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:09.531348 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.531269    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:09.531513 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.531365    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs podName:5dd6ce58-5b02-4e04-9463-38ccc4773b66 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:11.531320175 +0000 UTC m=+5.185040339 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs") pod "network-metrics-daemon-mh7bp" (UID: "5dd6ce58-5b02-4e04-9463-38ccc4773b66") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:09.531513 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.531384    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret podName:4c5e1758-ac02-4c25-8c56-7f229ac909b7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:11.531373955 +0000 UTC m=+5.185094114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret") pod "global-pull-secret-syncer-lgb42" (UID: "4c5e1758-ac02-4c25-8c56-7f229ac909b7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:09.632145 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.631480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:09.632145 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.631673    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:33:09.632145 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.631694    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:33:09.632145 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.631708    2569 projected.go:196] Error preparing data for projected volume kube-api-access-cmf4d for pod openshift-network-diagnostics/network-check-target-vbczg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:09.632145 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.631768    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d podName:57fdc049-4633-4a9d-a899-d13dae2f0b81 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:11.631749643 +0000 UTC m=+5.285469798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cmf4d" (UniqueName: "kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d") pod "network-check-target-vbczg" (UID: "57fdc049-4633-4a9d-a899-d13dae2f0b81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:09.944776 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.944709    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 23:28:07 +0000 UTC" deadline="2028-05-28 18:44:52.075673202 +0000 UTC"
Sep 10 23:33:09.944776 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.944773    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15019h11m42.130904481s"
Sep 10 23:33:09.972312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.972283    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:09.972566 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.972416    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:09.974400 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:09.972804    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:09.974400 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:09.972907    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:10.973158 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:10.973130    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:10.973738 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:10.973257    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:11.546413 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:11.546365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:11.546413 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:11.546414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:11.546623 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.546557    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:11.546623 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.546620    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs podName:5dd6ce58-5b02-4e04-9463-38ccc4773b66 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:15.546601152 +0000 UTC m=+9.200321308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs") pod "network-metrics-daemon-mh7bp" (UID: "5dd6ce58-5b02-4e04-9463-38ccc4773b66") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:11.547052 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.547032    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:11.547150 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.547084    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret podName:4c5e1758-ac02-4c25-8c56-7f229ac909b7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:15.547069643 +0000 UTC m=+9.200789801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret") pod "global-pull-secret-syncer-lgb42" (UID: "4c5e1758-ac02-4c25-8c56-7f229ac909b7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:11.647610 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:11.647549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:11.647787 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.647736    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:33:11.647787 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.647759    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:33:11.647787 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.647771    2569 projected.go:196] Error preparing data for projected volume kube-api-access-cmf4d for pod openshift-network-diagnostics/network-check-target-vbczg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:11.648072 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.647845    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d podName:57fdc049-4633-4a9d-a899-d13dae2f0b81 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:15.647814741 +0000 UTC m=+9.301534897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cmf4d" (UniqueName: "kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d") pod "network-check-target-vbczg" (UID: "57fdc049-4633-4a9d-a899-d13dae2f0b81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:11.971858 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:11.971824    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:11.972064 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.971981    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:11.972356 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:11.972334    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:11.972475 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:11.972451    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:12.972494 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:12.972371    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:12.972962 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:12.972525    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:13.973016 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:13.972984    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:13.973413 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:13.973031    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:13.973413 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:13.973129    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:13.973413 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:13.973221    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:14.972565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:14.972520    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:14.972738 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:14.972662    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:15.579206 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:15.579145    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:15.579206 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:15.579184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:15.579705 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.579331    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:15.579705 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.579398    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs podName:5dd6ce58-5b02-4e04-9463-38ccc4773b66 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:23.579377744 +0000 UTC m=+17.233097911 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs") pod "network-metrics-daemon-mh7bp" (UID: "5dd6ce58-5b02-4e04-9463-38ccc4773b66") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:15.579824 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.579764    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:15.579883 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.579832    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret podName:4c5e1758-ac02-4c25-8c56-7f229ac909b7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:23.579813268 +0000 UTC m=+17.233533428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret") pod "global-pull-secret-syncer-lgb42" (UID: "4c5e1758-ac02-4c25-8c56-7f229ac909b7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:15.680113 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:15.680010    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:15.680315 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.680162    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:33:15.680315 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.680186    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:33:15.680315 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.680198    2569 projected.go:196] Error preparing data for projected volume kube-api-access-cmf4d for pod openshift-network-diagnostics/network-check-target-vbczg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:15.680315 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.680258    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d podName:57fdc049-4633-4a9d-a899-d13dae2f0b81 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:23.680241089 +0000 UTC m=+17.333961254 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cmf4d" (UniqueName: "kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d") pod "network-check-target-vbczg" (UID: "57fdc049-4633-4a9d-a899-d13dae2f0b81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:15.972462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:15.972426    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:15.972641 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.972587    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:15.972718 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:15.972642    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:15.972802 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:15.972760    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:16.980443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:16.979600    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:16.980443 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:16.979778    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:17.971868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:17.971831    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:17.972053 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:17.971972    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:17.972053 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:17.972023    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:17.972164 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:17.972109    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:18.972386 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:18.972354    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:18.972841 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:18.972462    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:19.972673 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:19.972638    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:19.973162 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:19.972680    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:19.973162 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:19.972767    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:19.973162 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:19.972859    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:20.972776 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:20.972737    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:20.973258 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:20.972874    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:21.972615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:21.972577    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:21.972795 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:21.972628    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:21.972795 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:21.972705    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:21.973211 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:21.972856    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:22.972592 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:22.972561    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:22.972751 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:22.972668    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:23.638490 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:23.638449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:23.638490 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:23.638492    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:23.639032 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.638606    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:23.639032 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.638607    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:23.639032 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.638674    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs podName:5dd6ce58-5b02-4e04-9463-38ccc4773b66 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:39.638652777 +0000 UTC m=+33.292372931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs") pod "network-metrics-daemon-mh7bp" (UID: "5dd6ce58-5b02-4e04-9463-38ccc4773b66") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:23.639032 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.638716    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret podName:4c5e1758-ac02-4c25-8c56-7f229ac909b7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:39.638695749 +0000 UTC m=+33.292415909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret") pod "global-pull-secret-syncer-lgb42" (UID: "4c5e1758-ac02-4c25-8c56-7f229ac909b7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:23.739359 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:23.739320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:23.739589 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.739520    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 23:33:23.739589 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.739547    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 23:33:23.739589 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.739561    2569 projected.go:196] Error preparing data for projected volume kube-api-access-cmf4d for pod openshift-network-diagnostics/network-check-target-vbczg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:23.739796 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.739630    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d podName:57fdc049-4633-4a9d-a899-d13dae2f0b81 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:39.739611474 +0000 UTC m=+33.393331630 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cmf4d" (UniqueName: "kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d") pod "network-check-target-vbczg" (UID: "57fdc049-4633-4a9d-a899-d13dae2f0b81") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 23:33:23.972258 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:23.972179    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:23.972412 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.972306    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:23.972412 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:23.972366    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:23.972536 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:23.972500    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:24.972520 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:24.972481    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:24.973098 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:24.972590    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:25.972362 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:25.972324    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:25.972362 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:25.972364    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:25.972870 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:25.972454    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:25.972870 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:25.972572    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:26.973092 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:26.973070    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:26.973438 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:26.973164    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:27.163946 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:27.163898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-82p42" event={"ID":"c29833e6-791a-469b-931e-c40cda4afd2a","Type":"ContainerStarted","Data":"50f925588783da90f0c49ccf86b7927a0032b081007eb7a2c4d269c0d36e353b"}
Sep 10 23:33:27.165445 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:27.165416    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" event={"ID":"ed43c03ec92d5b25a955c89d154a23b6","Type":"ContainerStarted","Data":"454fecd1d2aea53c2b4b1773e887169a2c7bc7e76c897f0f764761b4f4e0498f"}
Sep 10 23:33:27.182653 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:27.182598    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-82p42" podStartSLOduration=1.509280157 podStartE2EDuration="20.182585215s" totalImagesPullingTime="18.673305058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:27.1824228 +0000 UTC m=+20.836142973" watchObservedRunningTime="2026-09-10 23:33:27.182585215 +0000 UTC m=+20.836305387"
Sep 10 23:33:27.195306 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:27.195265    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-154.ec2.internal" podStartSLOduration=20.195248073 podStartE2EDuration="20.195248073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:27.194757292 +0000 UTC m=+20.848477465" watchObservedRunningTime="2026-09-10 23:33:27.195248073 +0000 UTC m=+20.848968248"
Sep 10 23:33:27.972813 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:27.972625    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:27.972993 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:27.972672    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:27.972993 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:27.972890    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:27.973091 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:27.972990    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:28.169335 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.169303    2569 generic.go:356] "Generic (PLEG): container finished" podID="45284353ad5e38e301b33a8146d7b581" containerID="080a99081c147750a9d87b2d1d2fd8916ae9b0a1363849387a1f53cd9ce24112" exitCode=0
Sep 10 23:33:28.170152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.169376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" event={"ID":"45284353ad5e38e301b33a8146d7b581","Type":"ContainerDied","Data":"080a99081c147750a9d87b2d1d2fd8916ae9b0a1363849387a1f53cd9ce24112"}
Sep 10 23:33:28.171259 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.171233    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62" event={"ID":"b7412ae6-9d63-439a-b129-3036ca7ac110","Type":"ContainerStarted","Data":"136b271dad08b5da8f755ecb969626cef97ac779b7d460beab9f4f6cbe6a407c"}
Sep 10 23:33:28.175483 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.173296    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l297b" event={"ID":"29d288de-88fe-491c-a6fd-4edcd4cd5541","Type":"ContainerStarted","Data":"1ac1c5d3f74db90b2f70d3110a8df56f0167ad44b338168bfa127db02ece7c95"}
Sep 10 23:33:28.178596 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.178554    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vxkh" event={"ID":"1e74baf0-dfa1-45b8-87ba-8fb556a20495","Type":"ContainerStarted","Data":"ab1832d05e019e2749fab53bd9b6e1ca429a1bc4a6734eb1edc798efdd17724c"}
Sep 10 23:33:28.181820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.181792    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:33:28.182282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.182198    2569 generic.go:356] "Generic (PLEG): container finished" podID="2b0cbfcf-07ff-4607-939b-7eff0b21acf9" containerID="79319fc3a636979465af559d38ff541b80c5759d814c2888d27231f02c610525" exitCode=1
Sep 10 23:33:28.182282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.182267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"d33c67c544b50160ae066ab206d223dca0ed461e55a27b1e2f10d24faeefdbb2"}
Sep 10 23:33:28.182431 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.182347    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"a8176456bab6eabe07a66e81a5e0e1a645bd13a89e21dcd11d9dbcca567a23b8"}
Sep 10 23:33:28.182431 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.182400    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"0c23dd2ebef35a4bfe0f97f3c489d92e9d74523eaef66f3c68b42aa0dcbd4d24"}
Sep 10 23:33:28.182526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.182446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"7683bcebbb2b0bb7f503529df3d1087fb9703aa08ec10896e645df08ba7ebe70"}
Sep 10 23:33:28.182526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.182481    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"70e802340713de9b9e4bbd5b28ea6f1c5cc2964c883fb914defda52168252993"}
Sep 10 23:33:28.182526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.182515    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerDied","Data":"79319fc3a636979465af559d38ff541b80c5759d814c2888d27231f02c610525"}
Sep 10 23:33:28.184082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.183985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9qt4w" event={"ID":"85a41d6c-95e7-4a39-99e6-6b5ff81acfc3","Type":"ContainerStarted","Data":"64b311fe354c060a30b91a45b7fe6db174e1e35903b4fe2a6ae754d7a1f76469"}
Sep 10 23:33:28.185690 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.185664    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4fj9t" event={"ID":"f24f3d3d-137d-4393-b095-e651175daf4c","Type":"ContainerStarted","Data":"99f3edd716686b155f7d252882043064b2968a4ff5b2db7b6836dc3b2c7fbc87"}
Sep 10 23:33:28.187391 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.187368    2569 generic.go:356] "Generic (PLEG): container finished" podID="256ec37b-d6e3-436b-8fba-c5046b103926" containerID="a6ac37b2f3fb79b490811f77f91c6fa4dcbf52b9a6eb2a0b4f953c13c3a97b51" exitCode=0
Sep 10 23:33:28.187493 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.187456    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6th9" event={"ID":"256ec37b-d6e3-436b-8fba-c5046b103926","Type":"ContainerDied","Data":"a6ac37b2f3fb79b490811f77f91c6fa4dcbf52b9a6eb2a0b4f953c13c3a97b51"}
Sep 10 23:33:28.203015 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.202963    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7vxkh" podStartSLOduration=2.558758004 podStartE2EDuration="21.202919225s" totalImagesPullingTime="18.644161221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:28.202044467 +0000 UTC m=+21.855764652" watchObservedRunningTime="2026-09-10 23:33:28.202919225 +0000 UTC m=+21.856639400"
Sep 10 23:33:28.223261 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.223160    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l297b" podStartSLOduration=10.084784151 podStartE2EDuration="21.223138283s" totalImagesPullingTime="11.138354132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:28.223095452 +0000 UTC m=+21.876815625" watchObservedRunningTime="2026-09-10 23:33:28.223138283 +0000 UTC m=+21.876858459"
Sep 10 23:33:28.241744 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.241697    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9qt4w" podStartSLOduration=2.00700188 podStartE2EDuration="21.241678554s" totalImagesPullingTime="19.234676674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:28.241039929 +0000 UTC m=+21.894760103" watchObservedRunningTime="2026-09-10 23:33:28.241678554 +0000 UTC m=+21.895398729"
Sep 10 23:33:28.255494 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.255443    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4fj9t" podStartSLOduration=2.58898619 podStartE2EDuration="21.2554279s" totalImagesPullingTime="18.66644171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:28.255357849 +0000 UTC m=+21.909078024" watchObservedRunningTime="2026-09-10 23:33:28.2554279 +0000 UTC m=+21.909148074"
Sep 10 23:33:28.634230 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.634070    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 23:33:28.972685 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.972623    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:28.972865 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:28.972760    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:28.979347 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.979254    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T23:33:28.634226237Z","UUID":"72e19b6e-328d-4522-9aa9-baba62679462","Handler":null,"Name":"","Endpoint":""}
Sep 10 23:33:28.981067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.981043    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 23:33:28.981190 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:28.981080    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 23:33:29.192166 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:29.192124    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62" event={"ID":"b7412ae6-9d63-439a-b129-3036ca7ac110","Type":"ContainerStarted","Data":"8672652ac97d99417f133f2fc7ede2eb6503557fb20770fa9ede65524a28be51"}
Sep 10 23:33:29.193998 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:29.193967    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wz7cc" event={"ID":"d0dc0926-af13-418a-9c7d-b82d0afa5bd6","Type":"ContainerStarted","Data":"c24887f3446ef20eabbaea030ab36701f81e58a036b5d4af5f8b3ad1c2cef227"}
Sep 10 23:33:29.199481 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:29.199447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" event={"ID":"45284353ad5e38e301b33a8146d7b581","Type":"ContainerStarted","Data":"87a9ab139415b4d75a6c25ceb9163754a54e44757beab788e27bd6da3305e375"}
Sep 10 23:33:29.208192 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:29.208152    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wz7cc" podStartSLOduration=3.602258047 podStartE2EDuration="22.208137951s" totalImagesPullingTime="18.605879904s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:29.207707079 +0000 UTC m=+22.861427265" watchObservedRunningTime="2026-09-10 23:33:29.208137951 +0000 UTC m=+22.861858122"
Sep 10 23:33:29.222755 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:29.222663    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-154.ec2.internal" podStartSLOduration=22.222644834 podStartE2EDuration="22.222644834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:29.222404513 +0000 UTC m=+22.876124687" watchObservedRunningTime="2026-09-10 23:33:29.222644834 +0000 UTC m=+22.876365011"
Sep 10 23:33:29.972073 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:29.972033    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:29.972240 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:29.972033    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:29.972240 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:29.972171    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:29.972363 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:29.972235    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:30.068327 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.068240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:30.068975 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.068957    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:30.205006 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.204967    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62" event={"ID":"b7412ae6-9d63-439a-b129-3036ca7ac110","Type":"ContainerStarted","Data":"82a4de581dbd8e0f850e439808968a1ce52a86c7ab07a2a9d740d6d54c1d996d"}
Sep 10 23:33:30.208950 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.208918    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:33:30.209455 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.209427    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"56abdcb3949d86c69a587da08083f933e7e3c62b29f0ce4385fc9c81136fb51a"}
Sep 10 23:33:30.209671 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.209645    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:30.210343 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.210327    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l297b"
Sep 10 23:33:30.222555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.222504    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5ch62" podStartSLOduration=2.740678011 podStartE2EDuration="23.22246967s" totalImagesPullingTime="20.481791659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:30.222171542 +0000 UTC m=+23.875891714" watchObservedRunningTime="2026-09-10 23:33:30.22246967 +0000 UTC m=+23.876189846"
Sep 10 23:33:30.972914 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:30.972886    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:30.973079 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:30.973003    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:31.972499 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:31.972463    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:31.972957 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:31.972510    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:31.972957 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:31.972599    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:31.972957 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:31.972726    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:32.972735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:32.972693    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:32.973156 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:32.972836    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:33.220890 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:33.220826    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:33:33.221424 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:33.221294    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"81e5b15ec2f2801b76ee5250fe6c5cc43f30ed5614d0d3e071b135221b9d004b"}
Sep 10 23:33:33.221816 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:33.221576    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:33.221816 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:33.221652    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:33.222007 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:33.221811    2569 scope.go:117] "RemoveContainer" containerID="79319fc3a636979465af559d38ff541b80c5759d814c2888d27231f02c610525"
Sep 10 23:33:33.236456 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:33.236394    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:33.972163 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:33.971967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:33.972364 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:33.971992    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:33.972364 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:33.972269    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:33.972364 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:33.972309    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:34.227100 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:34.227028    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:33:34.227976 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:34.227773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" event={"ID":"2b0cbfcf-07ff-4607-939b-7eff0b21acf9","Type":"ContainerStarted","Data":"02420f96646544d97157b0035579d64cf3ac282c2cddafce987a7402a1ccfc67"}
Sep 10 23:33:34.230398 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:34.229429    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:34.232635 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:34.232612    2569 generic.go:356] "Generic (PLEG): container finished" podID="256ec37b-d6e3-436b-8fba-c5046b103926" containerID="5628dd8a500b668ff8cde15e54be180097351eccd1131084f0cd5a8170fabb07" exitCode=0
Sep 10 23:33:34.232715 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:34.232650    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6th9" event={"ID":"256ec37b-d6e3-436b-8fba-c5046b103926","Type":"ContainerDied","Data":"5628dd8a500b668ff8cde15e54be180097351eccd1131084f0cd5a8170fabb07"}
Sep 10 23:33:34.244992 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:34.244972    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:33:34.259548 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:34.259510    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2" podStartSLOduration=8.548115394 podStartE2EDuration="27.259497786s" totalImagesPullingTime="18.711382392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:34.258390707 +0000 UTC m=+27.912110879" watchObservedRunningTime="2026-09-10 23:33:34.259497786 +0000 UTC m=+27.913217959"
Sep 10 23:33:34.971909 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:34.971877    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:34.972069 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:34.972022    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:35.237243 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:35.237131    2569 generic.go:356] "Generic (PLEG): container finished" podID="256ec37b-d6e3-436b-8fba-c5046b103926" containerID="c1e7c1c1e36ba42661800003c1b49778005ef4585dc0bd9fca3fcf9c5c828ec3" exitCode=0
Sep 10 23:33:35.237243 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:35.237178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6th9" event={"ID":"256ec37b-d6e3-436b-8fba-c5046b103926","Type":"ContainerDied","Data":"c1e7c1c1e36ba42661800003c1b49778005ef4585dc0bd9fca3fcf9c5c828ec3"}
Sep 10 23:33:35.276131 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:35.276101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lgb42"]
Sep 10 23:33:35.276278 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:35.276211    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:35.276339 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:35.276294    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:35.279846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:35.279819    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vbczg"]
Sep 10 23:33:35.279997 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:35.279907    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:35.280061 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:35.280003    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:35.280512 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:35.280487    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mh7bp"]
Sep 10 23:33:35.280606 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:35.280586    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:35.280707 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:35.280675    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:36.242779 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:36.242686    2569 generic.go:356] "Generic (PLEG): container finished" podID="256ec37b-d6e3-436b-8fba-c5046b103926" containerID="15d00e38ffb96a9206a6b2a88a03d3df33d0e8e59c62a24251ef6ff7f2a24d01" exitCode=0
Sep 10 23:33:36.242779 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:36.242747    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6th9" event={"ID":"256ec37b-d6e3-436b-8fba-c5046b103926","Type":"ContainerDied","Data":"15d00e38ffb96a9206a6b2a88a03d3df33d0e8e59c62a24251ef6ff7f2a24d01"}
Sep 10 23:33:36.973251 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:36.973219    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:36.973441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:36.973330    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:36.973441 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:36.973351    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:36.973441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:36.973416    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:36.973441 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:36.973418    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:36.973660 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:36.973523    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:38.972534 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:38.972492    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:38.973031 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:38.972492    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:38.973031 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:38.972639    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lgb42" podUID="4c5e1758-ac02-4c25-8c56-7f229ac909b7"
Sep 10 23:33:38.973031 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:38.972492    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:38.973031 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:38.972721    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vbczg" podUID="57fdc049-4633-4a9d-a899-d13dae2f0b81"
Sep 10 23:33:38.973031 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:38.972875    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mh7bp" podUID="5dd6ce58-5b02-4e04-9463-38ccc4773b66"
Sep 10 23:33:39.087079 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.087051    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-154.ec2.internal" event="NodeReady"
Sep 10 23:33:39.087249 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.087202    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 23:33:39.136866 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.136756    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56c4d858d-8mjkj"]
Sep 10 23:33:39.232558 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.232464    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-8g9xq"]
Sep 10 23:33:39.232714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.232616    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.234824 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.234786    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 23:33:39.234984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.234970    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pdj78\""
Sep 10 23:33:39.235062 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.235043    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 23:33:39.235120 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.235080    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 23:33:39.242277 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.241012    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 23:33:39.246257 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.246239    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"]
Sep 10 23:33:39.246372 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.246360    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.248216 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.248194    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 23:33:39.248441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.248420    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.248441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.248433    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.248583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.248479    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 23:33:39.248668 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.248649    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-w4fhn\""
Sep 10 23:33:39.253545 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.253525    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 23:33:39.260086 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.260056    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"]
Sep 10 23:33:39.260202 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.260189    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.262511 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.262488    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 23:33:39.262757 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.262722    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 23:33:39.262871 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.262768    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pc6k9\""
Sep 10 23:33:39.262871 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.262798    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.262999 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.262923    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.272146 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.272120    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"]
Sep 10 23:33:39.272254 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.272243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:39.274006 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.273986    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 23:33:39.274113 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.274087    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-d6w6f\""
Sep 10 23:33:39.274233 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.274180    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.274233 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.274197    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.292194 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.292168    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-vm4gs"]
Sep 10 23:33:39.292323 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.292260    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.294674 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.294579    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.294674 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.294648    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 23:33:39.294674 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.294648    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zvr9h\""
Sep 10 23:33:39.294864 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.294828    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.294917 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.294885    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 23:33:39.310517 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.310492    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-8g9xq"]
Sep 10 23:33:39.310638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.310535    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-vm4gs"]
Sep 10 23:33:39.310638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.310557    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"]
Sep 10 23:33:39.310638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.310578    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"]
Sep 10 23:33:39.310638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.310596    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56c4d858d-8mjkj"]
Sep 10 23:33:39.310638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.310613    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"]
Sep 10 23:33:39.310638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.310635    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"]
Sep 10 23:33:39.310923 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.310595    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.313201 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.313021    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.313201 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.313031    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-6nn9p\""
Sep 10 23:33:39.313201 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.313021    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 23:33:39.313201 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.313150    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.313454 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.313398    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 23:33:39.318958 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.318925    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 23:33:39.324310 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.324273    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf"]
Sep 10 23:33:39.324427 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.324375    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.329151 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.327302    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 23:33:39.329151 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.327531    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lsrkg\""
Sep 10 23:33:39.329151 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.327721    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.329151 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.327748    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.329151 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.327865    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 23:33:39.344318 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.344281    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l297b_29d288de-88fe-491c-a6fd-4edcd4cd5541/konnectivity-agent/0.log" limit=1024
Sep 10 23:33:39.356207 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abd2e556-839b-4d11-8c58-566ff67e0fd9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.356318 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.356318 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84c23be9-3bf4-4ea4-803f-42b458c045ae-ca-trust-extracted\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.356318 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356296    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-bound-sa-token\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.356484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356346    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abd2e556-839b-4d11-8c58-566ff67e0fd9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.356484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356377    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-image-registry-private-configuration\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.356484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356421    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws5z9\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-kube-api-access-ws5z9\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.356484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356440    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e399181-306e-43df-a4e6-c35ecdd9881d-config\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.356484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsw5x\" (UniqueName: \"kubernetes.io/projected/abd2e556-839b-4d11-8c58-566ff67e0fd9-kube-api-access-wsw5x\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.356686 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356533    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.356686 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-certificates\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.356686 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c677bd31-57e0-4d9c-a478-f5d4d1373137-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.356686 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356649    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:39.356686 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356664    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e399181-306e-43df-a4e6-c35ecdd9881d-serving-cert\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.356894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-trusted-ca\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.356894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356710    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrrlt\" (UniqueName: \"kubernetes.io/projected/c677bd31-57e0-4d9c-a478-f5d4d1373137-kube-api-access-mrrlt\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.356894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq2zd\" (UniqueName: \"kubernetes.io/projected/0e399181-306e-43df-a4e6-c35ecdd9881d-kube-api-access-pq2zd\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.356894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356757    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-installation-pull-secrets\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.356894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356779    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clrgw\" (UniqueName: \"kubernetes.io/projected/2e488457-d28a-4070-8475-c5a0dbfd2b7c-kube-api-access-clrgw\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:39.356894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.356802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0e399181-306e-43df-a4e6-c35ecdd9881d-trusted-ca\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.361239 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.361218    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5hczz"]
Sep 10 23:33:39.361356 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.361339    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf"
Sep 10 23:33:39.363567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.363534    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gdqvb\""
Sep 10 23:33:39.363567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.363553    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.363724 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.363565    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.381409 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.381381    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g45l9"]
Sep 10 23:33:39.381535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.381519    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.383477 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.383459    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 23:33:39.383650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.383617    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 23:33:39.383772 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.383719    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n77qs\""
Sep 10 23:33:39.395010 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.394980    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6"]
Sep 10 23:33:39.395134 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.395066    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g45l9"
Sep 10 23:33:39.397630 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.397610    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.397998 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.397980    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wqzsr\""
Sep 10 23:33:39.398125 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.398006    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 23:33:39.398210 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.398068    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.410575 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.410555    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64d8787b64-rmldg"]
Sep 10 23:33:39.410696 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.410676    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6"
Sep 10 23:33:39.412641 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.412619    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.412721 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.412653    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.412785 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.412746    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-b4snj\""
Sep 10 23:33:39.422573 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.422554    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"]
Sep 10 23:33:39.422666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.422582    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf"]
Sep 10 23:33:39.422666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.422596    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6"]
Sep 10 23:33:39.422666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.422608    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5hczz"]
Sep 10 23:33:39.422666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.422628    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g45l9"]
Sep 10 23:33:39.422666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.422641    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64d8787b64-rmldg"]
Sep 10 23:33:39.422877 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.422774    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.424989 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.424969    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 23:33:39.425095 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.425041    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 23:33:39.425095 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.425066    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 23:33:39.425432 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.425414    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 23:33:39.425524 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.425414    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 23:33:39.425524 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.425418    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xfsbv\""
Sep 10 23:33:39.425628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.425547    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 23:33:39.434156 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.434128    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l297b_29d288de-88fe-491c-a6fd-4edcd4cd5541/konnectivity-agent/0.log" limit=1024
Sep 10 23:33:39.458127 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458099    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.458253 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458137    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a993d388-bd24-4154-b232-2665a25ecab8-trusted-ca-bundle\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.458253 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458169    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/178c4a3f-841b-4d6c-8817-78ec6dc25085-cert\") pod \"ingress-canary-g45l9\" (UID: \"178c4a3f-841b-4d6c-8817-78ec6dc25085\") " pod="openshift-ingress-canary/ingress-canary-g45l9"
Sep 10 23:33:39.458350 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.458257    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 23:33:39.458350 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.458275    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c4d858d-8mjkj: secret "image-registry-tls" not found
Sep 10 23:33:39.458350 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458286    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a993d388-bd24-4154-b232-2665a25ecab8-snapshots\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.458350 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458329    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e399181-306e-43df-a4e6-c35ecdd9881d-serving-cert\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.458535 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.458358    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls podName:84c23be9-3bf4-4ea4-803f-42b458c045ae nodeName:}" failed. No retries permitted until 2026-09-10 23:33:39.958337331 +0000 UTC m=+33.612057485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls") pod "image-registry-56c4d858d-8mjkj" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae") : secret "image-registry-tls" not found
Sep 10 23:33:39.458535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458454    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrrlt\" (UniqueName: \"kubernetes.io/projected/c677bd31-57e0-4d9c-a478-f5d4d1373137-kube-api-access-mrrlt\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.458535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458486    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2a3102-f700-476f-a46c-239843bd5356-serving-cert\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.458535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458518    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-trusted-ca\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.458740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq2zd\" (UniqueName: \"kubernetes.io/projected/0e399181-306e-43df-a4e6-c35ecdd9881d-kube-api-access-pq2zd\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.458740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458578    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aab137b-75bc-4e7d-b33a-590054eaa2f7-config-volume\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.458740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458602    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0e399181-306e-43df-a4e6-c35ecdd9881d-trusted-ca\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.458740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458625    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abd2e556-839b-4d11-8c58-566ff67e0fd9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.458740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458653    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a993d388-bd24-4154-b232-2665a25ecab8-service-ca-bundle\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.458740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458683    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a993d388-bd24-4154-b232-2665a25ecab8-tmp\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.458740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-image-registry-private-configuration\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.458740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws5z9\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-kube-api-access-ws5z9\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458773    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a993d388-bd24-4154-b232-2665a25ecab8-serving-cert\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-certificates\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458838    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458865    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458887    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aab137b-75bc-4e7d-b33a-590054eaa2f7-tmp-dir\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458910    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f2a3102-f700-476f-a46c-239843bd5356-config\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458958    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t57ll\" (UniqueName: \"kubernetes.io/projected/8aab137b-75bc-4e7d-b33a-590054eaa2f7-kube-api-access-t57ll\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.458999    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b6c7\" (UniqueName: \"kubernetes.io/projected/1d604ae4-a70a-4080-b57b-e1b80e804c4b-kube-api-access-8b6c7\") pod \"volume-data-source-validator-86695b7cc7-65dwf\" (UID: \"1d604ae4-a70a-4080-b57b-e1b80e804c4b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-installation-pull-secrets\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrgw\" (UniqueName: \"kubernetes.io/projected/2e488457-d28a-4070-8475-c5a0dbfd2b7c-kube-api-access-clrgw\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459092    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.459130 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459118    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb9cs\" (UniqueName: \"kubernetes.io/projected/a993d388-bd24-4154-b232-2665a25ecab8-kube-api-access-vb9cs\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84c23be9-3bf4-4ea4-803f-42b458c045ae-ca-trust-extracted\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459178    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-bound-sa-token\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459204    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abd2e556-839b-4d11-8c58-566ff67e0fd9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsw5x\" (UniqueName: \"kubernetes.io/projected/abd2e556-839b-4d11-8c58-566ff67e0fd9-kube-api-access-wsw5x\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459259    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5mn8\" (UniqueName: \"kubernetes.io/projected/178c4a3f-841b-4d6c-8817-78ec6dc25085-kube-api-access-q5mn8\") pod \"ingress-canary-g45l9\" (UID: \"178c4a3f-841b-4d6c-8817-78ec6dc25085\") " pod="openshift-ingress-canary/ingress-canary-g45l9"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c677bd31-57e0-4d9c-a478-f5d4d1373137-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckxnr\" (UniqueName: \"kubernetes.io/projected/7f2a3102-f700-476f-a46c-239843bd5356-kube-api-access-ckxnr\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459345    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e399181-306e-43df-a4e6-c35ecdd9881d-config\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.459544    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-trusted-ca\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.459619    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls podName:c677bd31-57e0-4d9c-a478-f5d4d1373137 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:39.959603248 +0000 UTC m=+33.613323416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-xpr5q" (UID: "c677bd31-57e0-4d9c-a478-f5d4d1373137") : secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0e399181-306e-43df-a4e6-c35ecdd9881d-trusted-ca\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.459714 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459542    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-certificates\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.460329 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.459749    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 23:33:39.460329 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.459880    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls podName:2e488457-d28a-4070-8475-c5a0dbfd2b7c nodeName:}" failed. No retries permitted until 2026-09-10 23:33:39.959866131 +0000 UTC m=+33.613586286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6pgws" (UID: "2e488457-d28a-4070-8475-c5a0dbfd2b7c") : secret "samples-operator-tls" not found
Sep 10 23:33:39.460329 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.459979    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0e399181-306e-43df-a4e6-c35ecdd9881d-config\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.460329 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.460069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/abd2e556-839b-4d11-8c58-566ff67e0fd9-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.460537 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.460356    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84c23be9-3bf4-4ea4-803f-42b458c045ae-ca-trust-extracted\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.460661 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.460639    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c677bd31-57e0-4d9c-a478-f5d4d1373137-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.463494 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.463406    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/abd2e556-839b-4d11-8c58-566ff67e0fd9-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.463596 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.463534    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-image-registry-private-configuration\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.463596 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.463536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-installation-pull-secrets\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.463708 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.463658    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0e399181-306e-43df-a4e6-c35ecdd9881d-serving-cert\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.467524 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.467498    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrrlt\" (UniqueName: \"kubernetes.io/projected/c677bd31-57e0-4d9c-a478-f5d4d1373137-kube-api-access-mrrlt\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.467836 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.467814    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsw5x\" (UniqueName: \"kubernetes.io/projected/abd2e556-839b-4d11-8c58-566ff67e0fd9-kube-api-access-wsw5x\") pod \"kube-storage-version-migrator-operator-6c8f84f897-cfdcc\" (UID: \"abd2e556-839b-4d11-8c58-566ff67e0fd9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.468222 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.468204    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq2zd\" (UniqueName: \"kubernetes.io/projected/0e399181-306e-43df-a4e6-c35ecdd9881d-kube-api-access-pq2zd\") pod \"console-operator-696884fb9c-8g9xq\" (UID: \"0e399181-306e-43df-a4e6-c35ecdd9881d\") " pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.468765 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.468740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrgw\" (UniqueName: \"kubernetes.io/projected/2e488457-d28a-4070-8475-c5a0dbfd2b7c-kube-api-access-clrgw\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:39.468853 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.468772    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-bound-sa-token\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.469124 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.469102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws5z9\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-kube-api-access-ws5z9\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.557021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.556923    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:39.560195 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwvc7\" (UniqueName: \"kubernetes.io/projected/312a8253-b91a-4bf3-81e6-6ef5ef0f2b37-kube-api-access-xwvc7\") pod \"network-check-source-6f8b768455-bd2z6\" (UID: \"312a8253-b91a-4bf3-81e6-6ef5ef0f2b37\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6"
Sep 10 23:33:39.560316 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560217    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a993d388-bd24-4154-b232-2665a25ecab8-serving-cert\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.560316 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560283    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.560422 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aab137b-75bc-4e7d-b33a-590054eaa2f7-tmp-dir\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.560422 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560374    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f2a3102-f700-476f-a46c-239843bd5356-config\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.560422 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t57ll\" (UniqueName: \"kubernetes.io/projected/8aab137b-75bc-4e7d-b33a-590054eaa2f7-kube-api-access-t57ll\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.560588 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.560588 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.560464    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:33:39.560588 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.560547    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls podName:8aab137b-75bc-4e7d-b33a-590054eaa2f7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:40.060527399 +0000 UTC m=+33.714247563 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls") pod "dns-default-5hczz" (UID: "8aab137b-75bc-4e7d-b33a-590054eaa2f7") : secret "dns-default-metrics-tls" not found
Sep 10 23:33:39.560735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560465    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b6c7\" (UniqueName: \"kubernetes.io/projected/1d604ae4-a70a-4080-b57b-e1b80e804c4b-kube-api-access-8b6c7\") pod \"volume-data-source-validator-86695b7cc7-65dwf\" (UID: \"1d604ae4-a70a-4080-b57b-e1b80e804c4b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf"
Sep 10 23:33:39.560735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb9cs\" (UniqueName: \"kubernetes.io/projected/a993d388-bd24-4154-b232-2665a25ecab8-kube-api-access-vb9cs\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.560735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5mn8\" (UniqueName: \"kubernetes.io/projected/178c4a3f-841b-4d6c-8817-78ec6dc25085-kube-api-access-q5mn8\") pod \"ingress-canary-g45l9\" (UID: \"178c4a3f-841b-4d6c-8817-78ec6dc25085\") " pod="openshift-ingress-canary/ingress-canary-g45l9"
Sep 10 23:33:39.560735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560685    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aab137b-75bc-4e7d-b33a-590054eaa2f7-tmp-dir\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.560947 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560903    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckxnr\" (UniqueName: \"kubernetes.io/projected/7f2a3102-f700-476f-a46c-239843bd5356-kube-api-access-ckxnr\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.561007 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.560985    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-default-certificate\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.561007 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.561102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561016    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7f2a3102-f700-476f-a46c-239843bd5356-config\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.561102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561062    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a993d388-bd24-4154-b232-2665a25ecab8-trusted-ca-bundle\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.561182 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/178c4a3f-841b-4d6c-8817-78ec6dc25085-cert\") pod \"ingress-canary-g45l9\" (UID: \"178c4a3f-841b-4d6c-8817-78ec6dc25085\") " pod="openshift-ingress-canary/ingress-canary-g45l9"
Sep 10 23:33:39.561182 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a993d388-bd24-4154-b232-2665a25ecab8-snapshots\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.561182 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561173    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-stats-auth\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.561323 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2a3102-f700-476f-a46c-239843bd5356-serving-cert\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.561323 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561216    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aab137b-75bc-4e7d-b33a-590054eaa2f7-config-volume\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.561323 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmlsg\" (UniqueName: \"kubernetes.io/projected/217cfaa5-caf3-4e47-837c-c8dc90c53f63-kube-api-access-cmlsg\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.561323 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a993d388-bd24-4154-b232-2665a25ecab8-service-ca-bundle\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.561323 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a993d388-bd24-4154-b232-2665a25ecab8-tmp\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.562072 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561665    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a993d388-bd24-4154-b232-2665a25ecab8-tmp\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.562072 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561913    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a993d388-bd24-4154-b232-2665a25ecab8-trusted-ca-bundle\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.562072 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.561970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8aab137b-75bc-4e7d-b33a-590054eaa2f7-config-volume\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.562299 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.562072    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a993d388-bd24-4154-b232-2665a25ecab8-service-ca-bundle\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.562359 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.562303    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a993d388-bd24-4154-b232-2665a25ecab8-snapshots\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.563017 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.562999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a993d388-bd24-4154-b232-2665a25ecab8-serving-cert\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.563828 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.563800    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/178c4a3f-841b-4d6c-8817-78ec6dc25085-cert\") pod \"ingress-canary-g45l9\" (UID: \"178c4a3f-841b-4d6c-8817-78ec6dc25085\") " pod="openshift-ingress-canary/ingress-canary-g45l9"
Sep 10 23:33:39.564030 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.563993    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7f2a3102-f700-476f-a46c-239843bd5356-serving-cert\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.569235 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.569186    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t57ll\" (UniqueName: \"kubernetes.io/projected/8aab137b-75bc-4e7d-b33a-590054eaa2f7-kube-api-access-t57ll\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:39.569326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.569280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckxnr\" (UniqueName: \"kubernetes.io/projected/7f2a3102-f700-476f-a46c-239843bd5356-kube-api-access-ckxnr\") pod \"service-ca-operator-cf7c776b7-rbn2q\" (UID: \"7f2a3102-f700-476f-a46c-239843bd5356\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.569717 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.569659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5mn8\" (UniqueName: \"kubernetes.io/projected/178c4a3f-841b-4d6c-8817-78ec6dc25085-kube-api-access-q5mn8\") pod \"ingress-canary-g45l9\" (UID: \"178c4a3f-841b-4d6c-8817-78ec6dc25085\") " pod="openshift-ingress-canary/ingress-canary-g45l9"
Sep 10 23:33:39.569797 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.569764    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"
Sep 10 23:33:39.570029 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.569987    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb9cs\" (UniqueName: \"kubernetes.io/projected/a993d388-bd24-4154-b232-2665a25ecab8-kube-api-access-vb9cs\") pod \"insights-operator-5844b856db-vm4gs\" (UID: \"a993d388-bd24-4154-b232-2665a25ecab8\") " pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.571451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.571432    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b6c7\" (UniqueName: \"kubernetes.io/projected/1d604ae4-a70a-4080-b57b-e1b80e804c4b-kube-api-access-8b6c7\") pod \"volume-data-source-validator-86695b7cc7-65dwf\" (UID: \"1d604ae4-a70a-4080-b57b-e1b80e804c4b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf"
Sep 10 23:33:39.621509 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.621483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-vm4gs"
Sep 10 23:33:39.637264 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.637239    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"
Sep 10 23:33:39.662232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.662208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.662354 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.662274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-default-certificate\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.662354 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.662302    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.662354 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.662324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:39.662354 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.662346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.662362    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.662397    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-stats-auth\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.662435    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:40.162414828 +0000 UTC m=+33.816134978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : secret "router-metrics-certs-default" not found
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.662444    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.662472    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:40.16245539 +0000 UTC m=+33.816175576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : configmap references non-existent config key: service-ca.crt
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.662496    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret podName:4c5e1758-ac02-4c25-8c56-7f229ac909b7 nodeName:}" failed. No retries permitted until 2026-09-10 23:34:11.662486284 +0000 UTC m=+65.316206438 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret") pod "global-pull-secret-syncer-lgb42" (UID: "4c5e1758-ac02-4c25-8c56-7f229ac909b7") : object "kube-system"/"original-pull-secret" not registered
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.662522    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.662531    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmlsg\" (UniqueName: \"kubernetes.io/projected/217cfaa5-caf3-4e47-837c-c8dc90c53f63-kube-api-access-cmlsg\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.662577    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwvc7\" (UniqueName: \"kubernetes.io/projected/312a8253-b91a-4bf3-81e6-6ef5ef0f2b37-kube-api-access-xwvc7\") pod \"network-check-source-6f8b768455-bd2z6\" (UID: \"312a8253-b91a-4bf3-81e6-6ef5ef0f2b37\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6"
Sep 10 23:33:39.662698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.662615    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs podName:5dd6ce58-5b02-4e04-9463-38ccc4773b66 nodeName:}" failed. No retries permitted until 2026-09-10 23:34:11.662599658 +0000 UTC m=+65.316319810 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs") pod "network-metrics-daemon-mh7bp" (UID: "5dd6ce58-5b02-4e04-9463-38ccc4773b66") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 23:33:39.665268 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.665249    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-stats-auth\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.665374 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.665275    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-default-certificate\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.670685 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.670661    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwvc7\" (UniqueName: \"kubernetes.io/projected/312a8253-b91a-4bf3-81e6-6ef5ef0f2b37-kube-api-access-xwvc7\") pod \"network-check-source-6f8b768455-bd2z6\" (UID: \"312a8253-b91a-4bf3-81e6-6ef5ef0f2b37\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6"
Sep 10 23:33:39.670842 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.670820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmlsg\" (UniqueName: \"kubernetes.io/projected/217cfaa5-caf3-4e47-837c-c8dc90c53f63-kube-api-access-cmlsg\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:39.671047 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.671009    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf"
Sep 10 23:33:39.704257 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.704234    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g45l9"
Sep 10 23:33:39.719378 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.719353    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6"
Sep 10 23:33:39.763517 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.763488    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:39.766384 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.766349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmf4d\" (UniqueName: \"kubernetes.io/projected/57fdc049-4633-4a9d-a899-d13dae2f0b81-kube-api-access-cmf4d\") pod \"network-check-target-vbczg\" (UID: \"57fdc049-4633-4a9d-a899-d13dae2f0b81\") " pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:39.965302 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.965260    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:39.965478 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.965329    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:39.965478 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:39.965408    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:39.965478 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.965412    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 23:33:39.965601 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.965499    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls podName:2e488457-d28a-4070-8475-c5a0dbfd2b7c nodeName:}" failed. No retries permitted until 2026-09-10 23:33:40.965473577 +0000 UTC m=+34.619193729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6pgws" (UID: "2e488457-d28a-4070-8475-c5a0dbfd2b7c") : secret "samples-operator-tls" not found
Sep 10 23:33:39.965601 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.965516    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:39.965601 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.965546    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 23:33:39.965601 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.965560    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c4d858d-8mjkj: secret "image-registry-tls" not found
Sep 10 23:33:39.965601 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.965567    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls podName:c677bd31-57e0-4d9c-a478-f5d4d1373137 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:40.965554465 +0000 UTC m=+34.619274616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-xpr5q" (UID: "c677bd31-57e0-4d9c-a478-f5d4d1373137") : secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:39.965601 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:39.965598    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls podName:84c23be9-3bf4-4ea4-803f-42b458c045ae nodeName:}" failed. No retries permitted until 2026-09-10 23:33:40.965587569 +0000 UTC m=+34.619307728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls") pod "image-registry-56c4d858d-8mjkj" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae") : secret "image-registry-tls" not found
Sep 10 23:33:40.066285 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.066240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:40.066716 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.066367    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:33:40.066716 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.066474    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls podName:8aab137b-75bc-4e7d-b33a-590054eaa2f7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:41.066421372 +0000 UTC m=+34.720141523 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls") pod "dns-default-5hczz" (UID: "8aab137b-75bc-4e7d-b33a-590054eaa2f7") : secret "dns-default-metrics-tls" not found
Sep 10 23:33:40.167204 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.167173    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:40.167616 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.167593    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:41.167323833 +0000 UTC m=+34.821043994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : configmap references non-existent config key: service-ca.crt
Sep 10 23:33:40.167723 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.167282    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:40.167875 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.167777    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:33:40.167875 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.167841    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:41.167826768 +0000 UTC m=+34.821546922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : secret "router-metrics-certs-default" not found
Sep 10 23:33:40.563697 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.563676    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf"]
Sep 10 23:33:40.566757 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.566700    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6"]
Sep 10 23:33:40.567348 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:40.567322    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d604ae4_a70a_4080_b57b_e1b80e804c4b.slice/crio-cd435d8549b314b44c8bc7418d1582a74ea2db401b68cb2a2bfa3db60174eb2d WatchSource:0}: Error finding container cd435d8549b314b44c8bc7418d1582a74ea2db401b68cb2a2bfa3db60174eb2d: Status 404 returned error can't find the container with id cd435d8549b314b44c8bc7418d1582a74ea2db401b68cb2a2bfa3db60174eb2d
Sep 10 23:33:40.569706 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.569667    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q"]
Sep 10 23:33:40.571465 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.571379    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g45l9"]
Sep 10 23:33:40.573395 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.572533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-8g9xq"]
Sep 10 23:33:40.574763 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.573994    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-vm4gs"]
Sep 10 23:33:40.574763 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:40.574583    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312a8253_b91a_4bf3_81e6_6ef5ef0f2b37.slice/crio-74ba82acca37247f7da813b3e32a8217ff4c7cae482fff13691651a74a09dfc2 WatchSource:0}: Error finding container 74ba82acca37247f7da813b3e32a8217ff4c7cae482fff13691651a74a09dfc2: Status 404 returned error can't find the container with id 74ba82acca37247f7da813b3e32a8217ff4c7cae482fff13691651a74a09dfc2
Sep 10 23:33:40.575179 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:40.575158    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e399181_306e_43df_a4e6_c35ecdd9881d.slice/crio-159dc3ac9e2eb9b20201cd58588d977e39993fc43ecfe95d95539213b265304f WatchSource:0}: Error finding container 159dc3ac9e2eb9b20201cd58588d977e39993fc43ecfe95d95539213b265304f: Status 404 returned error can't find the container with id 159dc3ac9e2eb9b20201cd58588d977e39993fc43ecfe95d95539213b265304f
Sep 10 23:33:40.576886 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:40.576846    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2a3102_f700_476f_a46c_239843bd5356.slice/crio-f18cc112f49bd259ffda0c15d2aa28f39456969b0ce86a55078b120504c7c830 WatchSource:0}: Error finding container f18cc112f49bd259ffda0c15d2aa28f39456969b0ce86a55078b120504c7c830: Status 404 returned error can't find the container with id f18cc112f49bd259ffda0c15d2aa28f39456969b0ce86a55078b120504c7c830
Sep 10 23:33:40.578742 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.578451    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc"]
Sep 10 23:33:40.972457 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.972425    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:33:40.972657 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.972427    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:33:40.972722 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.972439    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:40.974812 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.974783    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:40.974957 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.974886    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:40.974957 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.974897    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 23:33:40.974957 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.974911    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9ms7z\""
Sep 10 23:33:40.975129 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.974994    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:40.975129 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.975029    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 23:33:40.975129 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.975045    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c4d858d-8mjkj: secret "image-registry-tls" not found
Sep 10 23:33:40.975129 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.975057    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls podName:c677bd31-57e0-4d9c-a478-f5d4d1373137 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:42.97503721 +0000 UTC m=+36.628757365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-xpr5q" (UID: "c677bd31-57e0-4d9c-a478-f5d4d1373137") : secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:40.975129 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.974996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:40.975129 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.975070    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 23:33:40.975129 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.975091    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls podName:84c23be9-3bf4-4ea4-803f-42b458c045ae nodeName:}" failed. No retries permitted until 2026-09-10 23:33:42.975076103 +0000 UTC m=+36.628796269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls") pod "image-registry-56c4d858d-8mjkj" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae") : secret "image-registry-tls" not found
Sep 10 23:33:40.975129 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:40.975122    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls podName:2e488457-d28a-4070-8475-c5a0dbfd2b7c nodeName:}" failed. No retries permitted until 2026-09-10 23:33:42.975109437 +0000 UTC m=+36.628829605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6pgws" (UID: "2e488457-d28a-4070-8475-c5a0dbfd2b7c") : secret "samples-operator-tls" not found
Sep 10 23:33:40.975456 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.975293    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 23:33:40.975456 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.975302    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hvw46\""
Sep 10 23:33:40.992887 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:40.992871    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:41.076014 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.075986    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:41.076711 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:41.076114    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:33:41.076711 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:41.076162    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls podName:8aab137b-75bc-4e7d-b33a-590054eaa2f7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:43.076149099 +0000 UTC m=+36.729869250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls") pod "dns-default-5hczz" (UID: "8aab137b-75bc-4e7d-b33a-590054eaa2f7") : secret "dns-default-metrics-tls" not found
Sep 10 23:33:41.177820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.177510    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:41.177820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.177603    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:41.177820 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:41.177736    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:43.177716613 +0000 UTC m=+36.831436765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : configmap references non-existent config key: service-ca.crt
Sep 10 23:33:41.177820 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:41.177754    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:33:41.177820 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:41.177816    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:43.177795043 +0000 UTC m=+36.831515195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : secret "router-metrics-certs-default" not found
Sep 10 23:33:41.263381 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.263303    2569 generic.go:356] "Generic (PLEG): container finished" podID="256ec37b-d6e3-436b-8fba-c5046b103926" containerID="573237ee597344acb2efb51a60b65264cd9ff8a578e05503c3fc534e55a9ef77" exitCode=0
Sep 10 23:33:41.263560 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.263389    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6th9" event={"ID":"256ec37b-d6e3-436b-8fba-c5046b103926","Type":"ContainerDied","Data":"573237ee597344acb2efb51a60b65264cd9ff8a578e05503c3fc534e55a9ef77"}
Sep 10 23:33:41.267136 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.267107    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q" event={"ID":"7f2a3102-f700-476f-a46c-239843bd5356","Type":"ContainerStarted","Data":"f18cc112f49bd259ffda0c15d2aa28f39456969b0ce86a55078b120504c7c830"}
Sep 10 23:33:41.269581 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.269469    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6" event={"ID":"312a8253-b91a-4bf3-81e6-6ef5ef0f2b37","Type":"ContainerStarted","Data":"74ba82acca37247f7da813b3e32a8217ff4c7cae482fff13691651a74a09dfc2"}
Sep 10 23:33:41.272681 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.272657    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-vm4gs" event={"ID":"a993d388-bd24-4154-b232-2665a25ecab8","Type":"ContainerStarted","Data":"369862992b3de122d895f7f92317553139bd1de8e8164326b1f4849591509c11"}
Sep 10 23:33:41.274532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.274472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf" event={"ID":"1d604ae4-a70a-4080-b57b-e1b80e804c4b","Type":"ContainerStarted","Data":"cd435d8549b314b44c8bc7418d1582a74ea2db401b68cb2a2bfa3db60174eb2d"}
Sep 10 23:33:41.274766 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.274746    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vbczg"]
Sep 10 23:33:41.278554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.278504    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g45l9" event={"ID":"178c4a3f-841b-4d6c-8817-78ec6dc25085","Type":"ContainerStarted","Data":"931da40546cc18f1772e76aa2058585d585ed28ac604f9636e54af814bf45e4d"}
Sep 10 23:33:41.280779 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.280540    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq" event={"ID":"0e399181-306e-43df-a4e6-c35ecdd9881d","Type":"ContainerStarted","Data":"159dc3ac9e2eb9b20201cd58588d977e39993fc43ecfe95d95539213b265304f"}
Sep 10 23:33:41.281011 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:41.280981    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57fdc049_4633_4a9d_a899_d13dae2f0b81.slice/crio-48fdc87858eb90fc91da3be3e796dbe1dd036d3e9b0babe4ba5fa06618d29415 WatchSource:0}: Error finding container 48fdc87858eb90fc91da3be3e796dbe1dd036d3e9b0babe4ba5fa06618d29415: Status 404 returned error can't find the container with id 48fdc87858eb90fc91da3be3e796dbe1dd036d3e9b0babe4ba5fa06618d29415
Sep 10 23:33:41.282870 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.282814    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc" event={"ID":"abd2e556-839b-4d11-8c58-566ff67e0fd9","Type":"ContainerStarted","Data":"d851ba46895394660bce302d7cf0b81d76fa469943840c869e679d507e68fb75"}
Sep 10 23:33:41.917874 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.917837    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"]
Sep 10 23:33:41.936607 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.936553    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"]
Sep 10 23:33:41.936777 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.936681    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:41.939487 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.939456    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 10 23:33:41.939487 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.939480    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9pt47\""
Sep 10 23:33:41.939689 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:41.939582    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 10 23:33:42.085841 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.085682    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:42.085841 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.085761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:42.156555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.156497    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l297b_29d288de-88fe-491c-a6fd-4edcd4cd5541/konnectivity-agent/0.log" limit=1024
Sep 10 23:33:42.188737 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.187300    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:42.188737 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.187369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:42.188737 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.187581    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 23:33:42.188737 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.187654    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert podName:1378e9cb-d095-4c5c-ad0a-afd6b7e6f229 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:42.687633354 +0000 UTC m=+36.341353523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-tjtxx" (UID: "1378e9cb-d095-4c5c-ad0a-afd6b7e6f229") : secret "networking-console-plugin-cert" not found
Sep 10 23:33:42.188737 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.188601    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:42.252295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.252241    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l297b_29d288de-88fe-491c-a6fd-4edcd4cd5541/konnectivity-agent/0.log" limit=1024
Sep 10 23:33:42.295207 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.295167    2569 generic.go:356] "Generic (PLEG): container finished" podID="256ec37b-d6e3-436b-8fba-c5046b103926" containerID="751a82ad7074a365b5c49e11cd70948866e8dd7db6674730c86e56a5d341cf5a" exitCode=0
Sep 10 23:33:42.295415 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.295267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6th9" event={"ID":"256ec37b-d6e3-436b-8fba-c5046b103926","Type":"ContainerDied","Data":"751a82ad7074a365b5c49e11cd70948866e8dd7db6674730c86e56a5d341cf5a"}
Sep 10 23:33:42.298379 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.298291    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vbczg" event={"ID":"57fdc049-4633-4a9d-a899-d13dae2f0b81","Type":"ContainerStarted","Data":"48fdc87858eb90fc91da3be3e796dbe1dd036d3e9b0babe4ba5fa06618d29415"}
Sep 10 23:33:42.692345 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.692308    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:42.692635 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.692501    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 23:33:42.692635 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.692571    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert podName:1378e9cb-d095-4c5c-ad0a-afd6b7e6f229 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:43.692551809 +0000 UTC m=+37.346271980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-tjtxx" (UID: "1378e9cb-d095-4c5c-ad0a-afd6b7e6f229") : secret "networking-console-plugin-cert" not found
Sep 10 23:33:42.995877 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.995797    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:42.996070 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.995898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:42.996070 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:42.996002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:42.996070 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.996025    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:42.996201 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.996105    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls podName:c677bd31-57e0-4d9c-a478-f5d4d1373137 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:46.996081845 +0000 UTC m=+40.649802023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-xpr5q" (UID: "c677bd31-57e0-4d9c-a478-f5d4d1373137") : secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:42.996201 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.996132    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 23:33:42.996201 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.996182    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls podName:2e488457-d28a-4070-8475-c5a0dbfd2b7c nodeName:}" failed. No retries permitted until 2026-09-10 23:33:46.996165855 +0000 UTC m=+40.649886021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6pgws" (UID: "2e488457-d28a-4070-8475-c5a0dbfd2b7c") : secret "samples-operator-tls" not found
Sep 10 23:33:42.996201 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.996188    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 23:33:42.996332 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.996204    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c4d858d-8mjkj: secret "image-registry-tls" not found
Sep 10 23:33:42.996332 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:42.996243    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls podName:84c23be9-3bf4-4ea4-803f-42b458c045ae nodeName:}" failed. No retries permitted until 2026-09-10 23:33:46.996231051 +0000 UTC m=+40.649951213 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls") pod "image-registry-56c4d858d-8mjkj" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae") : secret "image-registry-tls" not found
Sep 10 23:33:43.097307 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:43.097270    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:43.098706 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:43.098685    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:33:43.098827 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:43.098757    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls podName:8aab137b-75bc-4e7d-b33a-590054eaa2f7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:47.098736422 +0000 UTC m=+40.752456578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls") pod "dns-default-5hczz" (UID: "8aab137b-75bc-4e7d-b33a-590054eaa2f7") : secret "dns-default-metrics-tls" not found
Sep 10 23:33:43.199857 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:43.199819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:43.200058 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:43.199921    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:43.200665 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:43.200118    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:47.200097766 +0000 UTC m=+40.853817921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : configmap references non-existent config key: service-ca.crt
Sep 10 23:33:43.200665 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:43.200550    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:33:43.200665 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:43.200604    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:47.200587254 +0000 UTC m=+40.854307409 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : secret "router-metrics-certs-default" not found
Sep 10 23:33:43.705427 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:43.705390    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:43.706110 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:43.705699    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 23:33:43.706110 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:43.705763    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert podName:1378e9cb-d095-4c5c-ad0a-afd6b7e6f229 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:45.705744933 +0000 UTC m=+39.359465089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-tjtxx" (UID: "1378e9cb-d095-4c5c-ad0a-afd6b7e6f229") : secret "networking-console-plugin-cert" not found
Sep 10 23:33:45.222043 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:45.221845    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l297b_29d288de-88fe-491c-a6fd-4edcd4cd5541/konnectivity-agent/0.log" limit=1024
Sep 10 23:33:45.322568 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:45.322527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l297b_29d288de-88fe-491c-a6fd-4edcd4cd5541/konnectivity-agent/0.log" limit=1024
Sep 10 23:33:45.724813 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:45.724776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:45.725028 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:45.724922    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 23:33:45.725028 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:45.725000    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert podName:1378e9cb-d095-4c5c-ad0a-afd6b7e6f229 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:49.724979027 +0000 UTC m=+43.378699189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-tjtxx" (UID: "1378e9cb-d095-4c5c-ad0a-afd6b7e6f229") : secret "networking-console-plugin-cert" not found
Sep 10 23:33:47.037150 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:47.037118    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:47.037165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:47.037215    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.037277    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.037316    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.037328    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c4d858d-8mjkj: secret "image-registry-tls" not found
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.037317    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.037348    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls podName:2e488457-d28a-4070-8475-c5a0dbfd2b7c nodeName:}" failed. No retries permitted until 2026-09-10 23:33:55.037332714 +0000 UTC m=+48.691052868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-6pgws" (UID: "2e488457-d28a-4070-8475-c5a0dbfd2b7c") : secret "samples-operator-tls" not found
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.037363    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls podName:84c23be9-3bf4-4ea4-803f-42b458c045ae nodeName:}" failed. No retries permitted until 2026-09-10 23:33:55.037356866 +0000 UTC m=+48.691077017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls") pod "image-registry-56c4d858d-8mjkj" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae") : secret "image-registry-tls" not found
Sep 10 23:33:47.037533 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.037401    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls podName:c677bd31-57e0-4d9c-a478-f5d4d1373137 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:55.037376278 +0000 UTC m=+48.691096450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-xpr5q" (UID: "c677bd31-57e0-4d9c-a478-f5d4d1373137") : secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:47.137804 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:47.137776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:47.137961 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.137915    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:33:47.138028 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.138005    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls podName:8aab137b-75bc-4e7d-b33a-590054eaa2f7 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:55.137984375 +0000 UTC m=+48.791704528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls") pod "dns-default-5hczz" (UID: "8aab137b-75bc-4e7d-b33a-590054eaa2f7") : secret "dns-default-metrics-tls" not found
Sep 10 23:33:47.238793 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:47.238755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:47.238989 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:47.238829    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:47.238989 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.238913    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:33:47.239097 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.239002    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:55.238981963 +0000 UTC m=+48.892702120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : secret "router-metrics-certs-default" not found
Sep 10 23:33:47.239097 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:47.239017    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:55.239011078 +0000 UTC m=+48.892731228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : configmap references non-existent config key: service-ca.crt
Sep 10 23:33:49.323284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.323067    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t6th9" event={"ID":"256ec37b-d6e3-436b-8fba-c5046b103926","Type":"ContainerStarted","Data":"5efee6557ad7079a4747ea093e1483749e81aca9cbf5cd471693c9cb57298ca9"}
Sep 10 23:33:49.325061 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.325035    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q" event={"ID":"7f2a3102-f700-476f-a46c-239843bd5356","Type":"ContainerStarted","Data":"f7f84bffde68965c87afe078f5203b6ea37ecfd97110b3d5e030765cf57fc5c2"}
Sep 10 23:33:49.326615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.326582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6" event={"ID":"312a8253-b91a-4bf3-81e6-6ef5ef0f2b37","Type":"ContainerStarted","Data":"2e3b6309975f9fa62f55cb95ee590438fdb6a26ec4a67a0e8c6ef3abe48d0eb1"}
Sep 10 23:33:49.328478 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.328455    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vbczg" event={"ID":"57fdc049-4633-4a9d-a899-d13dae2f0b81","Type":"ContainerStarted","Data":"3b55dbac9df569018fb561687465c34a33f8dad06242250ddbe9d332e22a2841"}
Sep 10 23:33:49.328604 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.328535    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:33:49.335326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.335290    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-vm4gs" event={"ID":"a993d388-bd24-4154-b232-2665a25ecab8","Type":"ContainerStarted","Data":"71f989ca205101da059ba48112cf9dd2cda0114fc3bb18936848783ad039d923"}
Sep 10 23:33:49.337032 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.336966    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf" event={"ID":"1d604ae4-a70a-4080-b57b-e1b80e804c4b","Type":"ContainerStarted","Data":"dc05a2864cfe61d762e9a28681cd83926809ccb360926fb7a23b133813699da3"}
Sep 10 23:33:49.338663 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.338642    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g45l9" event={"ID":"178c4a3f-841b-4d6c-8817-78ec6dc25085","Type":"ContainerStarted","Data":"8ff7e95d4164d6ca941848ffaa6380245632507bef935da429d65252923f24f6"}
Sep 10 23:33:49.340465 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.340445    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/0.log"
Sep 10 23:33:49.340568 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.340494    2569 generic.go:356] "Generic (PLEG): container finished" podID="0e399181-306e-43df-a4e6-c35ecdd9881d" containerID="033b7744c05b05c3916fd76046e885f5e8cd99107d323edc63877fd3490e0a90" exitCode=255
Sep 10 23:33:49.340630 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.340582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq" event={"ID":"0e399181-306e-43df-a4e6-c35ecdd9881d","Type":"ContainerDied","Data":"033b7744c05b05c3916fd76046e885f5e8cd99107d323edc63877fd3490e0a90"}
Sep 10 23:33:49.340733 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.340715    2569 scope.go:117] "RemoveContainer" containerID="033b7744c05b05c3916fd76046e885f5e8cd99107d323edc63877fd3490e0a90"
Sep 10 23:33:49.342311 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.342292    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc" event={"ID":"abd2e556-839b-4d11-8c58-566ff67e0fd9","Type":"ContainerStarted","Data":"5917ceddf5f145968abb05702b32a8437aa3df62f5f2d3de9ee27b431c2bffff"}
Sep 10 23:33:49.345284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.345246    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t6th9" podStartSLOduration=12.856251987 podStartE2EDuration="42.345234576s" totalImagesPullingTime="29.488982589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:49.344842939 +0000 UTC m=+42.998563115" watchObservedRunningTime="2026-09-10 23:33:49.345234576 +0000 UTC m=+42.998954749"
Sep 10 23:33:49.362404 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.362355    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-vm4gs" podStartSLOduration=15.224639661 podStartE2EDuration="23.362340931s" totalImagesPullingTime="8.13770127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:49.361976388 +0000 UTC m=+43.015696563" watchObservedRunningTime="2026-09-10 23:33:49.362340931 +0000 UTC m=+43.016061104"
Sep 10 23:33:49.377484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.377442    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-bd2z6" podStartSLOduration=15.220720514 podStartE2EDuration="23.377426523s" totalImagesPullingTime="8.156706009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:49.376239901 +0000 UTC m=+43.029960073" watchObservedRunningTime="2026-09-10 23:33:49.377426523 +0000 UTC m=+43.031146693"
Sep 10 23:33:49.411856 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.411792    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q" podStartSLOduration=15.289004379 podStartE2EDuration="23.411756017s" totalImagesPullingTime="8.122751638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:49.411244273 +0000 UTC m=+43.064964446" watchObservedRunningTime="2026-09-10 23:33:49.411756017 +0000 UTC m=+43.065476191"
Sep 10 23:33:49.412897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.412846    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vbczg" podStartSLOduration=34.977143405 podStartE2EDuration="42.412832704s" totalImagesPullingTime="7.435689299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:49.393378673 +0000 UTC m=+43.047098848" watchObservedRunningTime="2026-09-10 23:33:49.412832704 +0000 UTC m=+43.066552890"
Sep 10 23:33:49.429633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.429234    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-65dwf" podStartSLOduration=15.368624455 podStartE2EDuration="23.429214765s" totalImagesPullingTime="8.06059031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:49.428730202 +0000 UTC m=+43.082450358" watchObservedRunningTime="2026-09-10 23:33:49.429214765 +0000 UTC m=+43.082934940"
Sep 10 23:33:49.448820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.448739    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g45l9" podStartSLOduration=2.310517897 podStartE2EDuration="10.448706151s" totalImagesPullingTime="8.138188254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:49.448170708 +0000 UTC m=+43.101890882" watchObservedRunningTime="2026-09-10 23:33:49.448706151 +0000 UTC m=+43.102426325"
Sep 10 23:33:49.516583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.516519    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc" podStartSLOduration=15.379680133 podStartE2EDuration="23.516469623s" totalImagesPullingTime="8.13678949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:49.515490296 +0000 UTC m=+43.169210469" watchObservedRunningTime="2026-09-10 23:33:49.516469623 +0000 UTC m=+43.170189797"
Sep 10 23:33:49.557621 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.557575    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:49.557796 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.557633    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:49.759661 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:49.759626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:49.759842 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:49.759802    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 23:33:49.759899 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:49.759868    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert podName:1378e9cb-d095-4c5c-ad0a-afd6b7e6f229 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:57.759853763 +0000 UTC m=+51.413573925 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-tjtxx" (UID: "1378e9cb-d095-4c5c-ad0a-afd6b7e6f229") : secret "networking-console-plugin-cert" not found
Sep 10 23:33:50.349112 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.349083    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:33:50.349603 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.349583    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/0.log"
Sep 10 23:33:50.349679 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.349631    2569 generic.go:356] "Generic (PLEG): container finished" podID="0e399181-306e-43df-a4e6-c35ecdd9881d" containerID="57f43615c9f097c0e96b0a1ea639b9514eaeeeadb3127845766eb13bff693e31" exitCode=255
Sep 10 23:33:50.349902 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.349731    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq" event={"ID":"0e399181-306e-43df-a4e6-c35ecdd9881d","Type":"ContainerDied","Data":"57f43615c9f097c0e96b0a1ea639b9514eaeeeadb3127845766eb13bff693e31"}
Sep 10 23:33:50.349902 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.349811    2569 scope.go:117] "RemoveContainer" containerID="033b7744c05b05c3916fd76046e885f5e8cd99107d323edc63877fd3490e0a90"
Sep 10 23:33:50.350708 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.350160    2569 scope.go:117] "RemoveContainer" containerID="57f43615c9f097c0e96b0a1ea639b9514eaeeeadb3127845766eb13bff693e31"
Sep 10 23:33:50.350708 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:50.350354    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-8g9xq_openshift-console-operator(0e399181-306e-43df-a4e6-c35ecdd9881d)\"" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq" podUID="0e399181-306e-43df-a4e6-c35ecdd9881d"
Sep 10 23:33:50.426018 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.425972    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ghqvk"]
Sep 10 23:33:50.446354 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.446323    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ghqvk"]
Sep 10 23:33:50.446525 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.446411    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.448470 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.448446    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 23:33:50.448608 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.448447    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6vndn\""
Sep 10 23:33:50.448859 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.448842    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 23:33:50.568591 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.568548    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-data-volume\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.568759 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.568644    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s45mv\" (UniqueName: \"kubernetes.io/projected/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-api-access-s45mv\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.568759 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.568699    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.568841 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.568788    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-crio-socket\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.568876 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.568850    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.669370 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.669332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-crio-socket\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.669557 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.669415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.669557 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.669479    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-data-volume\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.669557 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.669519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s45mv\" (UniqueName: \"kubernetes.io/projected/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-api-access-s45mv\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.669557 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.669555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.669760 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.669556    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-crio-socket\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.669760 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:50.669567    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:33:50.669760 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:50.669642    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:51.169623436 +0000 UTC m=+44.823343586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : configmap "kube-rbac-proxy" not found
Sep 10 23:33:50.669760 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:50.669665    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:50.669760 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:50.669725    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:51.169709055 +0000 UTC m=+44.823429208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:50.669966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.669849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-data-volume\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.680367 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.680349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s45mv\" (UniqueName: \"kubernetes.io/projected/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-api-access-s45mv\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:50.789570 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.789537    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm"]
Sep 10 23:33:50.813597 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.813569    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm"]
Sep 10 23:33:50.813747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.813660    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm"
Sep 10 23:33:50.815929 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.815904    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 23:33:50.817006 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.816974    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5k98s\""
Sep 10 23:33:50.817006 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.817000    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 23:33:50.972473 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:50.972387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4lpl\" (UniqueName: \"kubernetes.io/projected/6d3dac95-bd90-44bf-bb9b-2837fcadcf58-kube-api-access-g4lpl\") pod \"migrator-8b4d697c5-m2hgm\" (UID: \"6d3dac95-bd90-44bf-bb9b-2837fcadcf58\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm"
Sep 10 23:33:51.073674 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.073639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4lpl\" (UniqueName: \"kubernetes.io/projected/6d3dac95-bd90-44bf-bb9b-2837fcadcf58-kube-api-access-g4lpl\") pod \"migrator-8b4d697c5-m2hgm\" (UID: \"6d3dac95-bd90-44bf-bb9b-2837fcadcf58\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm"
Sep 10 23:33:51.085615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.085582    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4lpl\" (UniqueName: \"kubernetes.io/projected/6d3dac95-bd90-44bf-bb9b-2837fcadcf58-kube-api-access-g4lpl\") pod \"migrator-8b4d697c5-m2hgm\" (UID: \"6d3dac95-bd90-44bf-bb9b-2837fcadcf58\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm"
Sep 10 23:33:51.122670 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.122638    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm"
Sep 10 23:33:51.175284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.175247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:51.175431 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:51.175367    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:33:51.175471 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:51.175444    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:52.175429607 +0000 UTC m=+45.829149766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : configmap "kube-rbac-proxy" not found
Sep 10 23:33:51.175517 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.175489    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:51.175640 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:51.175622    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:51.175682 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:51.175674    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:52.175663024 +0000 UTC m=+45.829383175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:51.356106 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.356027    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:33:51.356548 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.356433    2569 scope.go:117] "RemoveContainer" containerID="57f43615c9f097c0e96b0a1ea639b9514eaeeeadb3127845766eb13bff693e31"
Sep 10 23:33:51.356660 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:51.356639    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-8g9xq_openshift-console-operator(0e399181-306e-43df-a4e6-c35ecdd9881d)\"" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq" podUID="0e399181-306e-43df-a4e6-c35ecdd9881d"
Sep 10 23:33:51.394123 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.394097    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4fj9t_f24f3d3d-137d-4393-b095-e651175daf4c/dns-node-resolver/0.log"
Sep 10 23:33:51.401539 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:51.401214    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm"]
Sep 10 23:33:51.403879 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:51.403857    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3dac95_bd90_44bf_bb9b_2837fcadcf58.slice/crio-fab1e3604ab482ebd032632f7cf461f1be219330bd27eb9dfcbce2cef5f57d5e WatchSource:0}: Error finding container fab1e3604ab482ebd032632f7cf461f1be219330bd27eb9dfcbce2cef5f57d5e: Status 404 returned error can't find the container with id fab1e3604ab482ebd032632f7cf461f1be219330bd27eb9dfcbce2cef5f57d5e
Sep 10 23:33:52.187615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.187573    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:52.187803 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.187717    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:52.187803 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:52.187732    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:33:52.187930 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:52.187815    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:54.187790229 +0000 UTC m=+47.841510386 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : configmap "kube-rbac-proxy" not found
Sep 10 23:33:52.187930 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:52.187893    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:52.188050 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:52.187971    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:54.187952258 +0000 UTC m=+47.841672426 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:52.361888 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.361854    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm" event={"ID":"6d3dac95-bd90-44bf-bb9b-2837fcadcf58","Type":"ContainerStarted","Data":"fab1e3604ab482ebd032632f7cf461f1be219330bd27eb9dfcbce2cef5f57d5e"}
Sep 10 23:33:52.594728 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.594659    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vxkh_1e74baf0-dfa1-45b8-87ba-8fb556a20495/node-ca/0.log"
Sep 10 23:33:52.701079 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.701042    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-kwcj2"]
Sep 10 23:33:52.718459 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.718433    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-kwcj2"]
Sep 10 23:33:52.718620 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.718539    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:52.720680 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.720652    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-g2l8t\""
Sep 10 23:33:52.720802 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.720699    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 23:33:52.720802 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.720652    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 23:33:52.721530 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.721515    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 23:33:52.721530 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.721523    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 23:33:52.893461 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.893433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/01837fc4-f83c-4465-a330-afcb2aa4d419-signing-key\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:52.893618 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.893507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9x4z\" (UniqueName: \"kubernetes.io/projected/01837fc4-f83c-4465-a330-afcb2aa4d419-kube-api-access-n9x4z\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:52.893618 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.893588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/01837fc4-f83c-4465-a330-afcb2aa4d419-signing-cabundle\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:52.994155 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.994120    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/01837fc4-f83c-4465-a330-afcb2aa4d419-signing-cabundle\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:52.994317 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.994225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/01837fc4-f83c-4465-a330-afcb2aa4d419-signing-key\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:52.994317 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.994301    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9x4z\" (UniqueName: \"kubernetes.io/projected/01837fc4-f83c-4465-a330-afcb2aa4d419-kube-api-access-n9x4z\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:52.994796 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.994777    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/01837fc4-f83c-4465-a330-afcb2aa4d419-signing-cabundle\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:52.996680 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:52.996661    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/01837fc4-f83c-4465-a330-afcb2aa4d419-signing-key\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:53.002425 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:53.002405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9x4z\" (UniqueName: \"kubernetes.io/projected/01837fc4-f83c-4465-a330-afcb2aa4d419-kube-api-access-n9x4z\") pod \"service-ca-568869fdc5-kwcj2\" (UID: \"01837fc4-f83c-4465-a330-afcb2aa4d419\") " pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:53.027717 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:53.027693    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-kwcj2"
Sep 10 23:33:53.194748 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:53.194716    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g45l9_178c4a3f-841b-4d6c-8817-78ec6dc25085/serve-healthcheck-canary/0.log"
Sep 10 23:33:53.343493 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:53.343469    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-kwcj2"]
Sep 10 23:33:53.345966 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:33:53.345927    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01837fc4_f83c_4465_a330_afcb2aa4d419.slice/crio-5e2fa9db4c569bdb5f170db9b2dee5cdef61c80b040599e01ce332f99ec5db03 WatchSource:0}: Error finding container 5e2fa9db4c569bdb5f170db9b2dee5cdef61c80b040599e01ce332f99ec5db03: Status 404 returned error can't find the container with id 5e2fa9db4c569bdb5f170db9b2dee5cdef61c80b040599e01ce332f99ec5db03
Sep 10 23:33:53.370908 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:53.370877    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm" event={"ID":"6d3dac95-bd90-44bf-bb9b-2837fcadcf58","Type":"ContainerStarted","Data":"80ddce25bdd334ce1b2a4f50b8799d32e9caf76a11ebce1badafba8c6377951a"}
Sep 10 23:33:53.371325 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:53.370974    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm" event={"ID":"6d3dac95-bd90-44bf-bb9b-2837fcadcf58","Type":"ContainerStarted","Data":"fa5e4c3f8fc4add412cc773787000fffa4c0909bc98ee2bf5984bda2ef5d3471"}
Sep 10 23:33:53.372035 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:53.372009    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-kwcj2" event={"ID":"01837fc4-f83c-4465-a330-afcb2aa4d419","Type":"ContainerStarted","Data":"5e2fa9db4c569bdb5f170db9b2dee5cdef61c80b040599e01ce332f99ec5db03"}
Sep 10 23:33:53.393962 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:53.393903    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-m2hgm" podStartSLOduration=1.830128202 podStartE2EDuration="3.393887899s" totalImagesPullingTime="1.563759697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:53.392529039 +0000 UTC m=+47.046249215" watchObservedRunningTime="2026-09-10 23:33:53.393887899 +0000 UTC m=+47.047608110"
Sep 10 23:33:54.203200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:54.203164    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:54.203374 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:54.203286    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:54.203374 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:54.203294    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 23:33:54.203374 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:54.203361    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:58.203341489 +0000 UTC m=+51.857061662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : configmap "kube-rbac-proxy" not found
Sep 10 23:33:54.203502 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:54.203418    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:54.203502 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:54.203454    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:33:58.203443916 +0000 UTC m=+51.857164081 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:54.377807 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:54.377769    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-kwcj2" event={"ID":"01837fc4-f83c-4465-a330-afcb2aa4d419","Type":"ContainerStarted","Data":"0f2f1d7e3900c875cb9874e0d15145d9b95e1495075db8b1bf43eefc68d0ddb7"}
Sep 10 23:33:54.394606 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:54.394559    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-kwcj2" podStartSLOduration=2.39454102 podStartE2EDuration="2.39454102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:54.393660692 +0000 UTC m=+48.047380865" watchObservedRunningTime="2026-09-10 23:33:54.39454102 +0000 UTC m=+48.048261229"
Sep 10 23:33:55.110456 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.110420    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:55.110626 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.110475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:33:55.110626 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.110533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:33:55.110698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.110634    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:55.110698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.110646    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 23:33:55.110698 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.110658    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c4d858d-8mjkj: secret "image-registry-tls" not found
Sep 10 23:33:55.110783 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.110701    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls podName:c677bd31-57e0-4d9c-a478-f5d4d1373137 nodeName:}" failed. No retries permitted until 2026-09-10 23:34:11.110684023 +0000 UTC m=+64.764404183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-xpr5q" (UID: "c677bd31-57e0-4d9c-a478-f5d4d1373137") : secret "cluster-monitoring-operator-tls" not found
Sep 10 23:33:55.110783 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.110717    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls podName:84c23be9-3bf4-4ea4-803f-42b458c045ae nodeName:}" failed. No retries permitted until 2026-09-10 23:34:11.110710673 +0000 UTC m=+64.764430824 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls") pod "image-registry-56c4d858d-8mjkj" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae") : secret "image-registry-tls" not found
Sep 10 23:33:55.112747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.112726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2e488457-d28a-4070-8475-c5a0dbfd2b7c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-6pgws\" (UID: \"2e488457-d28a-4070-8475-c5a0dbfd2b7c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:55.181438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.181406    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"
Sep 10 23:33:55.211120 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.211091    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:33:55.211278 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.211248    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 23:33:55.211322 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.211312    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls podName:8aab137b-75bc-4e7d-b33a-590054eaa2f7 nodeName:}" failed. No retries permitted until 2026-09-10 23:34:11.211295752 +0000 UTC m=+64.865015921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls") pod "dns-default-5hczz" (UID: "8aab137b-75bc-4e7d-b33a-590054eaa2f7") : secret "dns-default-metrics-tls" not found
Sep 10 23:33:55.311738 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.311707    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:55.311898 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.311773    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:33:55.311898 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.311846    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 23:33:55.311993 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.311919    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:34:11.311904558 +0000 UTC m=+64.965624713 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : secret "router-metrics-certs-default" not found
Sep 10 23:33:55.311993 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:55.311946    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle podName:217cfaa5-caf3-4e47-837c-c8dc90c53f63 nodeName:}" failed. No retries permitted until 2026-09-10 23:34:11.311928202 +0000 UTC m=+64.965648353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle") pod "router-default-64d8787b64-rmldg" (UID: "217cfaa5-caf3-4e47-837c-c8dc90c53f63") : configmap references non-existent config key: service-ca.crt
Sep 10 23:33:55.480911 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:55.480880    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws"]
Sep 10 23:33:56.386691 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:56.386656    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws" event={"ID":"2e488457-d28a-4070-8475-c5a0dbfd2b7c","Type":"ContainerStarted","Data":"9e59047398a19a5de1fbc9cccde53566244c852e3466d043fb873804f15da007"}
Sep 10 23:33:57.835655 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:57.835579    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:33:57.836033 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:57.835738    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 10 23:33:57.836033 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:57.835822    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert podName:1378e9cb-d095-4c5c-ad0a-afd6b7e6f229 nodeName:}" failed. No retries permitted until 2026-09-10 23:34:13.835799255 +0000 UTC m=+67.489519425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-tjtxx" (UID: "1378e9cb-d095-4c5c-ad0a-afd6b7e6f229") : secret "networking-console-plugin-cert" not found
Sep 10 23:33:58.238602 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:58.238568    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:58.238793 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:58.238653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:58.238793 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:58.238754    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:58.238900 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:33:58.238805    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls podName:cc42875d-dbfc-46c1-b4db-925d5ffe4c98 nodeName:}" failed. No retries permitted until 2026-09-10 23:34:06.238792802 +0000 UTC m=+59.892512952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ghqvk" (UID: "cc42875d-dbfc-46c1-b4db-925d5ffe4c98") : secret "insights-runtime-extractor-tls" not found
Sep 10 23:33:58.239103 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:58.239082    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:33:58.397954 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:58.397907    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws" event={"ID":"2e488457-d28a-4070-8475-c5a0dbfd2b7c","Type":"ContainerStarted","Data":"816c118d396a35de4ff2b03ae46e1f9f5cf466a8e881ca5f2d7d8130657b2f03"}
Sep 10 23:33:58.398141 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:58.398029    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws" event={"ID":"2e488457-d28a-4070-8475-c5a0dbfd2b7c","Type":"ContainerStarted","Data":"55b7368fd73d9e5714f0e2d67e70210415e8a8fdef9b3b2b96a90ccc5a061576"}
Sep 10 23:33:58.418291 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:58.418231    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-6pgws" podStartSLOduration=30.405824632 podStartE2EDuration="32.418211962s" totalImagesPullingTime="2.01238733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:33:58.417582363 +0000 UTC m=+52.071302537" watchObservedRunningTime="2026-09-10 23:33:58.418211962 +0000 UTC m=+52.071932136"
Sep 10 23:33:59.557783 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:59.557744    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:59.558293 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:59.557802    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:33:59.558293 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:33:59.558144    2569 scope.go:117] "RemoveContainer" containerID="57f43615c9f097c0e96b0a1ea639b9514eaeeeadb3127845766eb13bff693e31"
Sep 10 23:34:00.405850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:00.405823    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:34:00.406107 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:00.405899    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq" event={"ID":"0e399181-306e-43df-a4e6-c35ecdd9881d","Type":"ContainerStarted","Data":"153dc3096ada421aac9b6b353e82c886f2ece3490d71e91b820a87f1160b08ac"}
Sep 10 23:34:00.406203 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:00.406184    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:34:00.410773 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:00.410755    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq"
Sep 10 23:34:00.424061 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:00.424009    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-8g9xq" podStartSLOduration=27.280323642 podStartE2EDuration="35.423993524s" totalImagesPullingTime="8.143669882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:00.422717044 +0000 UTC m=+54.076437217" watchObservedRunningTime="2026-09-10 23:34:00.423993524 +0000 UTC m=+54.077713699"
Sep 10 23:34:06.254120 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:06.254087    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zxbz2"
Sep 10 23:34:06.312222 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:06.312187    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:34:06.314485 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:06.314459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cc42875d-dbfc-46c1-b4db-925d5ffe4c98-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ghqvk\" (UID: \"cc42875d-dbfc-46c1-b4db-925d5ffe4c98\") " pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:34:06.358117 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:06.358081    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ghqvk"
Sep 10 23:34:06.630089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:06.630057    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ghqvk"]
Sep 10 23:34:06.632991 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:06.632969    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc42875d_dbfc_46c1_b4db_925d5ffe4c98.slice/crio-ac0b2c291f608952cf4fd93f2b03bc9060a33676ead4ceab7b71809a8e169d97 WatchSource:0}: Error finding container ac0b2c291f608952cf4fd93f2b03bc9060a33676ead4ceab7b71809a8e169d97: Status 404 returned error can't find the container with id ac0b2c291f608952cf4fd93f2b03bc9060a33676ead4ceab7b71809a8e169d97
Sep 10 23:34:07.431625 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:07.431589    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ghqvk" event={"ID":"cc42875d-dbfc-46c1-b4db-925d5ffe4c98","Type":"ContainerStarted","Data":"622d9d3dca8c7701979d16faec7816458b3e9d33a2a4e8014dcd59f8bbfd4246"}
Sep 10 23:34:07.432084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:07.431648    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ghqvk" event={"ID":"cc42875d-dbfc-46c1-b4db-925d5ffe4c98","Type":"ContainerStarted","Data":"450a4f6c93646899d12e9effa71e067eeebabf24845f134c9bdf19d0561adac0"}
Sep 10 23:34:07.432084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:07.431666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ghqvk" event={"ID":"cc42875d-dbfc-46c1-b4db-925d5ffe4c98","Type":"ContainerStarted","Data":"ac0b2c291f608952cf4fd93f2b03bc9060a33676ead4ceab7b71809a8e169d97"}
Sep 10 23:34:09.440449 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:09.440412    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ghqvk" event={"ID":"cc42875d-dbfc-46c1-b4db-925d5ffe4c98","Type":"ContainerStarted","Data":"da07f55da47a0ee2360f2a91029a9f2f7ca42fa2df11dbb666926e1aa5716eec"}
Sep 10 23:34:09.456795 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:09.456748    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ghqvk" podStartSLOduration=17.440575561 podStartE2EDuration="19.456734146s" totalImagesPullingTime="2.016158585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:09.456102934 +0000 UTC m=+63.109823107" watchObservedRunningTime="2026-09-10 23:34:09.456734146 +0000 UTC m=+63.110454319"
Sep 10 23:34:11.159825 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.159786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:34:11.160221 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.159868    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:34:11.162439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.162401    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"image-registry-56c4d858d-8mjkj\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") " pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:34:11.162734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.162707    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c677bd31-57e0-4d9c-a478-f5d4d1373137-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-xpr5q\" (UID: \"c677bd31-57e0-4d9c-a478-f5d4d1373137\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:34:11.260882 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.260844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:34:11.263163 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.263145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8aab137b-75bc-4e7d-b33a-590054eaa2f7-metrics-tls\") pod \"dns-default-5hczz\" (UID: \"8aab137b-75bc-4e7d-b33a-590054eaa2f7\") " pod="openshift-dns/dns-default-5hczz"
Sep 10 23:34:11.348125 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.348094    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pdj78\""
Sep 10 23:34:11.357055 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.357024    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:34:11.361533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.361505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:11.361673 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.361656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:11.362259 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.362237    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/217cfaa5-caf3-4e47-837c-c8dc90c53f63-service-ca-bundle\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:11.364052 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.364033    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/217cfaa5-caf3-4e47-837c-c8dc90c53f63-metrics-certs\") pod \"router-default-64d8787b64-rmldg\" (UID: \"217cfaa5-caf3-4e47-837c-c8dc90c53f63\") " pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:11.405204 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.405168    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zvr9h\""
Sep 10 23:34:11.413209 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.413155    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"
Sep 10 23:34:11.493383 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.493361    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n77qs\""
Sep 10 23:34:11.502016 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.501991    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5hczz"
Sep 10 23:34:11.535057 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.535031    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xfsbv\""
Sep 10 23:34:11.543816 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.543787    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:11.627137 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.627114    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56c4d858d-8mjkj"]
Sep 10 23:34:11.629621 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:11.629593    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84c23be9_3bf4_4ea4_803f_42b458c045ae.slice/crio-d540a1f5d7432fd7d0c4d4b47da72716ed399b2e2efd14c55f68f17403353742 WatchSource:0}: Error finding container d540a1f5d7432fd7d0c4d4b47da72716ed399b2e2efd14c55f68f17403353742: Status 404 returned error can't find the container with id d540a1f5d7432fd7d0c4d4b47da72716ed399b2e2efd14c55f68f17403353742
Sep 10 23:34:11.664351 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.664284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:34:11.664351 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.664319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:34:11.666475 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.666453    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 23:34:11.666583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.666453    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 23:34:11.676856 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.676831    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4c5e1758-ac02-4c25-8c56-7f229ac909b7-original-pull-secret\") pod \"global-pull-secret-syncer-lgb42\" (UID: \"4c5e1758-ac02-4c25-8c56-7f229ac909b7\") " pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:34:11.676987 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.676965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5dd6ce58-5b02-4e04-9463-38ccc4773b66-metrics-certs\") pod \"network-metrics-daemon-mh7bp\" (UID: \"5dd6ce58-5b02-4e04-9463-38ccc4773b66\") " pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:34:11.682809 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.682628    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q"]
Sep 10 23:34:11.686950 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:11.686914    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc677bd31_57e0_4d9c_a478_f5d4d1373137.slice/crio-69eeb8624a827d8a6644b56945db8911be097f9ef0a25b85e2e1f96735e2b568 WatchSource:0}: Error finding container 69eeb8624a827d8a6644b56945db8911be097f9ef0a25b85e2e1f96735e2b568: Status 404 returned error can't find the container with id 69eeb8624a827d8a6644b56945db8911be097f9ef0a25b85e2e1f96735e2b568
Sep 10 23:34:11.771573 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.771551    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5hczz"]
Sep 10 23:34:11.773826 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:11.773802    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aab137b_75bc_4e7d_b33a_590054eaa2f7.slice/crio-3ddd0a7ab5f515f7b7dc079090dba5f9693f2c351589308c32371cfa3fdb17b6 WatchSource:0}: Error finding container 3ddd0a7ab5f515f7b7dc079090dba5f9693f2c351589308c32371cfa3fdb17b6: Status 404 returned error can't find the container with id 3ddd0a7ab5f515f7b7dc079090dba5f9693f2c351589308c32371cfa3fdb17b6
Sep 10 23:34:11.836385 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.836359    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64d8787b64-rmldg"]
Sep 10 23:34:11.838772 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:11.838744    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217cfaa5_caf3_4e47_837c_c8dc90c53f63.slice/crio-4be1b01c3c3fc56c25b1f6d5cec3ea4ca360b27fd3efb1dc5408f3d1d2b1f648 WatchSource:0}: Error finding container 4be1b01c3c3fc56c25b1f6d5cec3ea4ca360b27fd3efb1dc5408f3d1d2b1f648: Status 404 returned error can't find the container with id 4be1b01c3c3fc56c25b1f6d5cec3ea4ca360b27fd3efb1dc5408f3d1d2b1f648
Sep 10 23:34:11.882993 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.882966    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lgb42"
Sep 10 23:34:11.890349 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.890325    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9ms7z\""
Sep 10 23:34:11.899004 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:11.898986    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mh7bp"
Sep 10 23:34:12.176424 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.176324    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lgb42"]
Sep 10 23:34:12.179619 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:12.179588    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5e1758_ac02_4c25_8c56_7f229ac909b7.slice/crio-6435ce41f8e5da919438d4ba1140b32b9e02c87322b7f1ac6abdd8e6d6816adc WatchSource:0}: Error finding container 6435ce41f8e5da919438d4ba1140b32b9e02c87322b7f1ac6abdd8e6d6816adc: Status 404 returned error can't find the container with id 6435ce41f8e5da919438d4ba1140b32b9e02c87322b7f1ac6abdd8e6d6816adc
Sep 10 23:34:12.206440 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.206300    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mh7bp"]
Sep 10 23:34:12.211692 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:12.211659    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dd6ce58_5b02_4e04_9463_38ccc4773b66.slice/crio-779a4e87ffd72b28347b5775f4474564b671909c20aa8871b962a65be5d83715 WatchSource:0}: Error finding container 779a4e87ffd72b28347b5775f4474564b671909c20aa8871b962a65be5d83715: Status 404 returned error can't find the container with id 779a4e87ffd72b28347b5775f4474564b671909c20aa8871b962a65be5d83715
Sep 10 23:34:12.451794 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.451713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q" event={"ID":"c677bd31-57e0-4d9c-a478-f5d4d1373137","Type":"ContainerStarted","Data":"69eeb8624a827d8a6644b56945db8911be097f9ef0a25b85e2e1f96735e2b568"}
Sep 10 23:34:12.453222 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.453190    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mh7bp" event={"ID":"5dd6ce58-5b02-4e04-9463-38ccc4773b66","Type":"ContainerStarted","Data":"779a4e87ffd72b28347b5775f4474564b671909c20aa8871b962a65be5d83715"}
Sep 10 23:34:12.454462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.454438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5hczz" event={"ID":"8aab137b-75bc-4e7d-b33a-590054eaa2f7","Type":"ContainerStarted","Data":"3ddd0a7ab5f515f7b7dc079090dba5f9693f2c351589308c32371cfa3fdb17b6"}
Sep 10 23:34:12.455560 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.455538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lgb42" event={"ID":"4c5e1758-ac02-4c25-8c56-7f229ac909b7","Type":"ContainerStarted","Data":"6435ce41f8e5da919438d4ba1140b32b9e02c87322b7f1ac6abdd8e6d6816adc"}
Sep 10 23:34:12.456985 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.456960    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64d8787b64-rmldg" event={"ID":"217cfaa5-caf3-4e47-837c-c8dc90c53f63","Type":"ContainerStarted","Data":"847e40d77eb0177db215faafbc2bc5af85b667b1d825b8f10b13b4cd7c6660bc"}
Sep 10 23:34:12.457077 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.457000    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64d8787b64-rmldg" event={"ID":"217cfaa5-caf3-4e47-837c-c8dc90c53f63","Type":"ContainerStarted","Data":"4be1b01c3c3fc56c25b1f6d5cec3ea4ca360b27fd3efb1dc5408f3d1d2b1f648"}
Sep 10 23:34:12.458466 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.458444    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj" event={"ID":"84c23be9-3bf4-4ea4-803f-42b458c045ae","Type":"ContainerStarted","Data":"a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc"}
Sep 10 23:34:12.458563 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.458485    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj" event={"ID":"84c23be9-3bf4-4ea4-803f-42b458c045ae","Type":"ContainerStarted","Data":"d540a1f5d7432fd7d0c4d4b47da72716ed399b2e2efd14c55f68f17403353742"}
Sep 10 23:34:12.458623 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.458581    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:34:12.477453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.477397    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64d8787b64-rmldg" podStartSLOduration=46.47737678 podStartE2EDuration="46.47737678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:12.476467635 +0000 UTC m=+66.130187809" watchObservedRunningTime="2026-09-10 23:34:12.47737678 +0000 UTC m=+66.131096954"
Sep 10 23:34:12.511692 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.511641    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj" podStartSLOduration=65.511619753 podStartE2EDuration="1m5.511619753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:12.510427082 +0000 UTC m=+66.164147257" watchObservedRunningTime="2026-09-10 23:34:12.511619753 +0000 UTC m=+66.165339927"
Sep 10 23:34:12.544946 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.544827    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:12.547774 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.547752    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:12.639795 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.639754    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"]
Sep 10 23:34:12.661289 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.661256    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"]
Sep 10 23:34:12.664100 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.663152    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"
Sep 10 23:34:12.668555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.667303    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 23:34:12.668555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.667532    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 23:34:12.668555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.667824    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pvpvn\""
Sep 10 23:34:12.668555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.668115    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 23:34:12.673437 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.673036    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 23:34:12.675821 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.675800    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"]
Sep 10 23:34:12.678448 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.676217    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.682386 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.682363    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 23:34:12.682484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.682387    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 23:34:12.682654 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.682637    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 23:34:12.682761 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.682748    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 23:34:12.701595 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.701568    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"]
Sep 10 23:34:12.701719 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.701608    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"]
Sep 10 23:34:12.701719 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.701625    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"]
Sep 10 23:34:12.701849 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.701776    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.704031 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.704011    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 23:34:12.738403 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.738314    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-vbzh6"]
Sep 10 23:34:12.753845 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.753820    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-vbzh6"
Sep 10 23:34:12.756708 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.756459    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 23:34:12.756708 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.756596    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-vbzh6"]
Sep 10 23:34:12.756952 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.756856    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 23:34:12.757143 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.757095    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4g9qp\""
Sep 10 23:34:12.775857 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.775816    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.776033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.775865    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e901f759-95a9-47a3-8714-639296f17c51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9bc5949b4-zpnqw\" (UID: \"e901f759-95a9-47a3-8714-639296f17c51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"
Sep 10 23:34:12.776033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.775893    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpzrj\" (UniqueName: \"kubernetes.io/projected/e901f759-95a9-47a3-8714-639296f17c51-kube-api-access-bpzrj\") pod \"managed-serviceaccount-addon-agent-9bc5949b4-zpnqw\" (UID: \"e901f759-95a9-47a3-8714-639296f17c51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"
Sep 10 23:34:12.776033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.775954    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.776033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.775982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da063e95-c863-4c8d-820e-aa2bd559b343-tmp\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.776033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.776013    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7vhs\" (UniqueName: \"kubernetes.io/projected/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-kube-api-access-j7vhs\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.776438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.776071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/da063e95-c863-4c8d-820e-aa2bd559b343-klusterlet-config\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.776438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.776096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-hub\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.776438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.776180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-ca\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.776438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.776240    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.776438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.776273    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd6k8\" (UniqueName: \"kubernetes.io/projected/da063e95-c863-4c8d-820e-aa2bd559b343-kube-api-access-jd6k8\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.877666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877630    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.877666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877672    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e901f759-95a9-47a3-8714-639296f17c51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9bc5949b4-zpnqw\" (UID: \"e901f759-95a9-47a3-8714-639296f17c51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"
Sep 10 23:34:12.877918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpzrj\" (UniqueName: \"kubernetes.io/projected/e901f759-95a9-47a3-8714-639296f17c51-kube-api-access-bpzrj\") pod \"managed-serviceaccount-addon-agent-9bc5949b4-zpnqw\" (UID: \"e901f759-95a9-47a3-8714-639296f17c51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"
Sep 10 23:34:12.877918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877726    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhj56\" (UniqueName: \"kubernetes.io/projected/0fcbe7f7-c497-4112-9b12-a96a3be661b0-kube-api-access-vhj56\") pod \"downloads-76f99584f4-vbzh6\" (UID: \"0fcbe7f7-c497-4112-9b12-a96a3be661b0\") " pod="openshift-console/downloads-76f99584f4-vbzh6"
Sep 10 23:34:12.877918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.877918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da063e95-c863-4c8d-820e-aa2bd559b343-tmp\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.877918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7vhs\" (UniqueName: \"kubernetes.io/projected/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-kube-api-access-j7vhs\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.877918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/da063e95-c863-4c8d-820e-aa2bd559b343-klusterlet-config\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.877918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877916    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-hub\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.878295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-ca\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.878295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.877989    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.878295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.878013    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd6k8\" (UniqueName: \"kubernetes.io/projected/da063e95-c863-4c8d-820e-aa2bd559b343-kube-api-access-jd6k8\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.878438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.878407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da063e95-c863-4c8d-820e-aa2bd559b343-tmp\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.880901 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.880874    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.881317 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.881253    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.881317 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.881308    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-ca\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.882389 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.882367    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-hub\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.882602 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.882557    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e901f759-95a9-47a3-8714-639296f17c51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9bc5949b4-zpnqw\" (UID: \"e901f759-95a9-47a3-8714-639296f17c51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"
Sep 10 23:34:12.883326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.883306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/da063e95-c863-4c8d-820e-aa2bd559b343-klusterlet-config\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.885318 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.885278    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.887965 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.887901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7vhs\" (UniqueName: \"kubernetes.io/projected/bcd23a4b-cb3a-45c2-9afa-9b1f71014011-kube-api-access-j7vhs\") pod \"cluster-proxy-proxy-agent-5c578ff498-x9kvx\" (UID: \"bcd23a4b-cb3a-45c2-9afa-9b1f71014011\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:12.888499 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.888468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpzrj\" (UniqueName: \"kubernetes.io/projected/e901f759-95a9-47a3-8714-639296f17c51-kube-api-access-bpzrj\") pod \"managed-serviceaccount-addon-agent-9bc5949b4-zpnqw\" (UID: \"e901f759-95a9-47a3-8714-639296f17c51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"
Sep 10 23:34:12.888927 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.888904    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd6k8\" (UniqueName: \"kubernetes.io/projected/da063e95-c863-4c8d-820e-aa2bd559b343-kube-api-access-jd6k8\") pod \"klusterlet-addon-workmgr-6475f55bfb-5hzmp\" (UID: \"da063e95-c863-4c8d-820e-aa2bd559b343\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:12.979514 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.979428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhj56\" (UniqueName: \"kubernetes.io/projected/0fcbe7f7-c497-4112-9b12-a96a3be661b0-kube-api-access-vhj56\") pod \"downloads-76f99584f4-vbzh6\" (UID: \"0fcbe7f7-c497-4112-9b12-a96a3be661b0\") " pod="openshift-console/downloads-76f99584f4-vbzh6"
Sep 10 23:34:12.989710 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.989630    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"
Sep 10 23:34:12.991074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:12.991053    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhj56\" (UniqueName: \"kubernetes.io/projected/0fcbe7f7-c497-4112-9b12-a96a3be661b0-kube-api-access-vhj56\") pod \"downloads-76f99584f4-vbzh6\" (UID: \"0fcbe7f7-c497-4112-9b12-a96a3be661b0\") " pod="openshift-console/downloads-76f99584f4-vbzh6"
Sep 10 23:34:13.000833 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:13.000810    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"
Sep 10 23:34:13.024582 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:13.024551    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:13.067583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:13.067545    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-vbzh6"
Sep 10 23:34:13.463336 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:13.463305    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:13.464952 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:13.464897    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64d8787b64-rmldg"
Sep 10 23:34:13.887513 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:13.887476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:34:13.890454 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:13.890405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1378e9cb-d095-4c5c-ad0a-afd6b7e6f229-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-tjtxx\" (UID: \"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:34:14.062452 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:14.062369    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9pt47\""
Sep 10 23:34:14.070915 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:14.070862    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"
Sep 10 23:34:15.473634 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.473453    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q" event={"ID":"c677bd31-57e0-4d9c-a478-f5d4d1373137","Type":"ContainerStarted","Data":"0c185575b0996f09a3d2781a063d3716b090c0e0769ef701aeedb6d74104f1fd"}
Sep 10 23:34:15.478548 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.477088    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx"]
Sep 10 23:34:15.481764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.481445    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mh7bp" event={"ID":"5dd6ce58-5b02-4e04-9463-38ccc4773b66","Type":"ContainerStarted","Data":"296b04a00683463bc98da50807622e2f632f46f34b41ade6bb07182ed68c59dc"}
Sep 10 23:34:15.483567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.483542    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5hczz" event={"ID":"8aab137b-75bc-4e7d-b33a-590054eaa2f7","Type":"ContainerStarted","Data":"ce9e760dd8b84d1f2ee6c1be996169597fce18186bca54e93bba5bbf2960526b"}
Sep 10 23:34:15.486298 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.486277    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw"]
Sep 10 23:34:15.496350 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.496292    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"]
Sep 10 23:34:15.498453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.498432    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx"]
Sep 10 23:34:15.514163 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.514080    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-xpr5q" podStartSLOduration=46.040482345 podStartE2EDuration="49.514065703s" totalImagesPullingTime="3.473583358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:15.512357353 +0000 UTC m=+69.166077525" watchObservedRunningTime="2026-09-10 23:34:15.514065703 +0000 UTC m=+69.167785875"
Sep 10 23:34:15.516795 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:15.516772    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-vbzh6"]
Sep 10 23:34:16.950765 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:16.950736    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode901f759_95a9_47a3_8714_639296f17c51.slice/crio-33287b2f554f1fc1b4040abe8394fb6ebabf34149b46c269cdd20e6eb52d1cc0 WatchSource:0}: Error finding container 33287b2f554f1fc1b4040abe8394fb6ebabf34149b46c269cdd20e6eb52d1cc0: Status 404 returned error can't find the container with id 33287b2f554f1fc1b4040abe8394fb6ebabf34149b46c269cdd20e6eb52d1cc0
Sep 10 23:34:16.952047 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:16.951692    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda063e95_c863_4c8d_820e_aa2bd559b343.slice/crio-1de91b8a413acdb9ec79c799010bc1692fe3bd92d09f301ac3cae22f2910dc14 WatchSource:0}: Error finding container 1de91b8a413acdb9ec79c799010bc1692fe3bd92d09f301ac3cae22f2910dc14: Status 404 returned error can't find the container with id 1de91b8a413acdb9ec79c799010bc1692fe3bd92d09f301ac3cae22f2910dc14
Sep 10 23:34:16.954292 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:16.954258    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1378e9cb_d095_4c5c_ad0a_afd6b7e6f229.slice/crio-64b8ac92d2d9ae9b11a9ed9327a9e806395372f833cf7f54e4c6eab4c8c0f163 WatchSource:0}: Error finding container 64b8ac92d2d9ae9b11a9ed9327a9e806395372f833cf7f54e4c6eab4c8c0f163: Status 404 returned error can't find the container with id 64b8ac92d2d9ae9b11a9ed9327a9e806395372f833cf7f54e4c6eab4c8c0f163
Sep 10 23:34:16.954579 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:16.954538    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fcbe7f7_c497_4112_9b12_a96a3be661b0.slice/crio-396d3b33031cd3862eff0ca5fc10858db40872b06a3c34e79a43de9d5f5afaec WatchSource:0}: Error finding container 396d3b33031cd3862eff0ca5fc10858db40872b06a3c34e79a43de9d5f5afaec: Status 404 returned error can't find the container with id 396d3b33031cd3862eff0ca5fc10858db40872b06a3c34e79a43de9d5f5afaec
Sep 10 23:34:17.494583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.494544    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mh7bp" event={"ID":"5dd6ce58-5b02-4e04-9463-38ccc4773b66","Type":"ContainerStarted","Data":"0ada65e0e8e8e6316acabd2856b47730353f5c0d103c5298595c6db8d5f00c90"}
Sep 10 23:34:17.496479 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.496452    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5hczz" event={"ID":"8aab137b-75bc-4e7d-b33a-590054eaa2f7","Type":"ContainerStarted","Data":"886c4b98d6d51c0f18d11773953a8581ada546f88aebe661488a456af225a7a8"}
Sep 10 23:34:17.496624 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.496512    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5hczz"
Sep 10 23:34:17.498227 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.498202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lgb42" event={"ID":"4c5e1758-ac02-4c25-8c56-7f229ac909b7","Type":"ContainerStarted","Data":"910f01c4dfee65479dc03f420d317d3b1b589adcbc3f7f9a08e1e9f94ef611aa"}
Sep 10 23:34:17.499607 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.499581    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw" event={"ID":"e901f759-95a9-47a3-8714-639296f17c51","Type":"ContainerStarted","Data":"33287b2f554f1fc1b4040abe8394fb6ebabf34149b46c269cdd20e6eb52d1cc0"}
Sep 10 23:34:17.500723 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.500700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx" event={"ID":"bcd23a4b-cb3a-45c2-9afa-9b1f71014011","Type":"ContainerStarted","Data":"69c3f453c60888d5f4d6b197f1cc07b85fe07cf6f2697c156196a0101af5f1d8"}
Sep 10 23:34:17.501810 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.501787    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp" event={"ID":"da063e95-c863-4c8d-820e-aa2bd559b343","Type":"ContainerStarted","Data":"1de91b8a413acdb9ec79c799010bc1692fe3bd92d09f301ac3cae22f2910dc14"}
Sep 10 23:34:17.504570 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.504540    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx" event={"ID":"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229","Type":"ContainerStarted","Data":"64b8ac92d2d9ae9b11a9ed9327a9e806395372f833cf7f54e4c6eab4c8c0f163"}
Sep 10 23:34:17.505701 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.505682    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-vbzh6" event={"ID":"0fcbe7f7-c497-4112-9b12-a96a3be661b0","Type":"ContainerStarted","Data":"396d3b33031cd3862eff0ca5fc10858db40872b06a3c34e79a43de9d5f5afaec"}
Sep 10 23:34:17.510570 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.510532    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mh7bp" podStartSLOduration=67.553633116 podStartE2EDuration="1m10.510518721s" totalImagesPullingTime="2.956885605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:17.509369502 +0000 UTC m=+71.163089675" watchObservedRunningTime="2026-09-10 23:34:17.510518721 +0000 UTC m=+71.164238894"
Sep 10 23:34:17.523338 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.522960    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lgb42" podStartSLOduration=65.37692857 podStartE2EDuration="1m10.522922971s" totalImagesPullingTime="5.145994401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 23:33:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:17.52260126 +0000 UTC m=+71.176321434" watchObservedRunningTime="2026-09-10 23:34:17.522922971 +0000 UTC m=+71.176643144"
Sep 10 23:34:17.540092 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:17.540046    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5hczz" podStartSLOduration=35.14558013 podStartE2EDuration="38.540032619s" totalImagesPullingTime="3.394452489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:17.539253636 +0000 UTC m=+71.192973810" watchObservedRunningTime="2026-09-10 23:34:17.540032619 +0000 UTC m=+71.193752791"
Sep 10 23:34:20.353900 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:20.353868    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vbczg"
Sep 10 23:34:23.566431 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:23.566308    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw" event={"ID":"e901f759-95a9-47a3-8714-639296f17c51","Type":"ContainerStarted","Data":"9d2c16e88a19ea2b017a2eab4166237c6b759afeb2b62b58837bb5ba2cd4ee40"}
Sep 10 23:34:23.569781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:23.569750    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp" event={"ID":"da063e95-c863-4c8d-820e-aa2bd559b343","Type":"ContainerStarted","Data":"0575ad6b4f82df3cd6e6840a86f2fdbf603670d44ad51b198140634e5de69aea"}
Sep 10 23:34:23.569986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:23.569903    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:23.571573 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:23.571551    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp"
Sep 10 23:34:23.572089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:23.572067    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx" event={"ID":"1378e9cb-d095-4c5c-ad0a-afd6b7e6f229","Type":"ContainerStarted","Data":"f115f88c55d60ac4aa57a79b188158be985c566f7ffdb2f1056a5467ba670fd6"}
Sep 10 23:34:23.582325 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:23.582217    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9bc5949b4-zpnqw" podStartSLOduration=5.195683324 podStartE2EDuration="11.582194513s" totalImagesPullingTime="6.386511189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:23.580696307 +0000 UTC m=+77.234416481" watchObservedRunningTime="2026-09-10 23:34:23.582194513 +0000 UTC m=+77.235914691"
Sep 10 23:34:23.606456 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:23.606387    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-tjtxx" podStartSLOduration=37.924864489 podStartE2EDuration="42.606366459s" totalImagesPullingTime="4.68150197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:33:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:23.602418649 +0000 UTC m=+77.256138823" watchObservedRunningTime="2026-09-10 23:34:23.606366459 +0000 UTC m=+77.260086633"
Sep 10 23:34:23.622695 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:23.622635    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6475f55bfb-5hzmp" podStartSLOduration=5.162323711 podStartE2EDuration="11.622614245s" totalImagesPullingTime="6.460290534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:23.620956695 +0000 UTC m=+77.274676869" watchObservedRunningTime="2026-09-10 23:34:23.622614245 +0000 UTC m=+77.276334418"
Sep 10 23:34:25.586385 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:25.586300    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx" event={"ID":"bcd23a4b-cb3a-45c2-9afa-9b1f71014011","Type":"ContainerStarted","Data":"a0705f027dcff0d761fcd4da8ccb3bae0cd1dd88e64a1d6f6711b93e16cb79cf"}
Sep 10 23:34:25.586385 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:25.586347    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx" event={"ID":"bcd23a4b-cb3a-45c2-9afa-9b1f71014011","Type":"ContainerStarted","Data":"fce4c4666862176c066391a6f4e7012ac2446b6f566191f6246afe7bda3ec369"}
Sep 10 23:34:25.586385 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:25.586364    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx" event={"ID":"bcd23a4b-cb3a-45c2-9afa-9b1f71014011","Type":"ContainerStarted","Data":"229586bb45cf197b74d904c96d072fac043a1a070bdcb8308f552a2a18473e66"}
Sep 10 23:34:25.605562 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:25.605518    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5c578ff498-x9kvx" podStartSLOduration=5.385076926 podStartE2EDuration="13.605501472s" totalImagesPullingTime="8.220424546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:25.604348074 +0000 UTC m=+79.258068250" watchObservedRunningTime="2026-09-10 23:34:25.605501472 +0000 UTC m=+79.259221644"
Sep 10 23:34:27.532868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:27.532817    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5hczz"
Sep 10 23:34:28.249490 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.248977    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-djsm9"]
Sep 10 23:34:28.254522 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.254340    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.258562 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.258540    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 23:34:28.260432 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.260410    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s2m6r\""
Sep 10 23:34:28.260537 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.260481    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 23:34:28.260644 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.260631    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 23:34:28.260854 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.260837    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 23:34:28.327560 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327522    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lspc\" (UniqueName: \"kubernetes.io/projected/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-kube-api-access-9lspc\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.327560 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327576    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-wtmp\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.327817 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327602    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.327817 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327645    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.327817 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327671    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-sys\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.327817 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327695    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-metrics-client-ca\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.328035 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-textfile\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.328035 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327895    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-root\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.328035 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.327950    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-tls\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.429361 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429321    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-root\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.429567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-tls\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.429567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429420    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lspc\" (UniqueName: \"kubernetes.io/projected/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-kube-api-access-9lspc\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.429567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429450    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-wtmp\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.429567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429449    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-root\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.429567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429477    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.429567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429523    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.429567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429551    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-sys\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.430145 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:34:28.429569    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 23:34:28.430145 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:34:28.429642    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-tls podName:072fedc4-764f-4c72-a3f7-5a5b50bb6f9a nodeName:}" failed. No retries permitted until 2026-09-10 23:34:28.929620719 +0000 UTC m=+82.583340885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-tls") pod "node-exporter-djsm9" (UID: "072fedc4-764f-4c72-a3f7-5a5b50bb6f9a") : secret "node-exporter-tls" not found
Sep 10 23:34:28.430145 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-metrics-client-ca\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.430145 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429742    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-textfile\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.430145 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.429908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-sys\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.430145 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.430064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-wtmp\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.430451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.430186    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-textfile\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.430451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.430306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.430553 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.430488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-metrics-client-ca\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.433856 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.433834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.441539 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.440964    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lspc\" (UniqueName: \"kubernetes.io/projected/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-kube-api-access-9lspc\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.936284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.936244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-tls\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:28.939102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:28.939079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/072fedc4-764f-4c72-a3f7-5a5b50bb6f9a-node-exporter-tls\") pod \"node-exporter-djsm9\" (UID: \"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a\") " pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:29.167969 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:29.167820    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-djsm9"
Sep 10 23:34:31.363121 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:31.363070    2569 patch_prober.go:28] interesting pod/image-registry-56c4d858d-8mjkj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 23:34:31.363625 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:31.363138    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj" podUID="84c23be9-3bf4-4ea4-803f-42b458c045ae" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 23:34:33.468031 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:33.467994    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:34:34.958989 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:34:34.958961    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072fedc4_764f_4c72_a3f7_5a5b50bb6f9a.slice/crio-24c5580807a0d7cf553a239196658353442980e6b3db94ddadc28ecc937d70c5 WatchSource:0}: Error finding container 24c5580807a0d7cf553a239196658353442980e6b3db94ddadc28ecc937d70c5: Status 404 returned error can't find the container with id 24c5580807a0d7cf553a239196658353442980e6b3db94ddadc28ecc937d70c5
Sep 10 23:34:35.636820 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:35.636775    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-djsm9" event={"ID":"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a","Type":"ContainerStarted","Data":"24c5580807a0d7cf553a239196658353442980e6b3db94ddadc28ecc937d70c5"}
Sep 10 23:34:35.640473 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:35.639398    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-vbzh6" event={"ID":"0fcbe7f7-c497-4112-9b12-a96a3be661b0","Type":"ContainerStarted","Data":"a06525495931843103a8393c86dee2e5a47e6c3baa6a35c907fcddd4b0937da6"}
Sep 10 23:34:35.640473 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:35.640423    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-vbzh6"
Sep 10 23:34:35.658039 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:35.657680    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-vbzh6" podStartSLOduration=5.381370164 podStartE2EDuration="23.657655889s" totalImagesPullingTime="18.276285725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:34:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:35.655122833 +0000 UTC m=+89.308843006" watchObservedRunningTime="2026-09-10 23:34:35.657655889 +0000 UTC m=+89.311376062"
Sep 10 23:34:35.660447 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:35.660420    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-vbzh6"
Sep 10 23:34:35.731998 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:35.731296    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56c4d858d-8mjkj"]
Sep 10 23:34:37.650158 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:37.650116    2569 generic.go:356] "Generic (PLEG): container finished" podID="072fedc4-764f-4c72-a3f7-5a5b50bb6f9a" containerID="d142f72f343eddf823647f512229fdd2b39f2146f3a3f9f78e762463d4a9591b" exitCode=0
Sep 10 23:34:37.650619 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:37.650223    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-djsm9" event={"ID":"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a","Type":"ContainerDied","Data":"d142f72f343eddf823647f512229fdd2b39f2146f3a3f9f78e762463d4a9591b"}
Sep 10 23:34:38.658327 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:38.658289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-djsm9" event={"ID":"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a","Type":"ContainerStarted","Data":"6c7643a7db6031fbe3df9ba7fda209e724a3d9c094f628fb98942ffdfdc2f6bb"}
Sep 10 23:34:38.658761 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:38.658352    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-djsm9" event={"ID":"072fedc4-764f-4c72-a3f7-5a5b50bb6f9a","Type":"ContainerStarted","Data":"bac43624f6ca927bd8fb5ef454c1ee52458512b9f4e8e7504c565d98f8ad2c8b"}
Sep 10 23:34:38.679973 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:34:38.679909    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-djsm9" podStartSLOduration=8.825806019 podStartE2EDuration="10.679892811s" totalImagesPullingTime="1.768873412s" totalInitContainerRuntime="85.21338ms" isStatefulPod=true podCreationTimestamp="2026-09-10 23:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:34:38.678533457 +0000 UTC m=+92.332253630" watchObservedRunningTime="2026-09-10 23:34:38.679892811 +0000 UTC m=+92.333612984"
Sep 10 23:35:00.765402 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:00.765337    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj" podUID="84c23be9-3bf4-4ea4-803f-42b458c045ae" containerName="registry" containerID="cri-o://a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc" gracePeriod=30
Sep 10 23:35:01.066026 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.065993    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:35:01.122110 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122077    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-certificates\") pod \"84c23be9-3bf4-4ea4-803f-42b458c045ae\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") "
Sep 10 23:35:01.122289 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122133    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") pod \"84c23be9-3bf4-4ea4-803f-42b458c045ae\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") "
Sep 10 23:35:01.122289 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122159    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-bound-sa-token\") pod \"84c23be9-3bf4-4ea4-803f-42b458c045ae\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") "
Sep 10 23:35:01.122289 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122281    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-installation-pull-secrets\") pod \"84c23be9-3bf4-4ea4-803f-42b458c045ae\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") "
Sep 10 23:35:01.122453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122324    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84c23be9-3bf4-4ea4-803f-42b458c045ae-ca-trust-extracted\") pod \"84c23be9-3bf4-4ea4-803f-42b458c045ae\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") "
Sep 10 23:35:01.122453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122372    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-trusted-ca\") pod \"84c23be9-3bf4-4ea4-803f-42b458c045ae\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") "
Sep 10 23:35:01.122453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122416    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws5z9\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-kube-api-access-ws5z9\") pod \"84c23be9-3bf4-4ea4-803f-42b458c045ae\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") "
Sep 10 23:35:01.122612 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122459    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-image-registry-private-configuration\") pod \"84c23be9-3bf4-4ea4-803f-42b458c045ae\" (UID: \"84c23be9-3bf4-4ea4-803f-42b458c045ae\") "
Sep 10 23:35:01.122612 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122540    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "84c23be9-3bf4-4ea4-803f-42b458c045ae" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:35:01.123002 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122803    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "84c23be9-3bf4-4ea4-803f-42b458c045ae" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:35:01.123002 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.122894    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-certificates\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:35:01.124771 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.124749    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "84c23be9-3bf4-4ea4-803f-42b458c045ae" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:35:01.124888 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.124866    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "84c23be9-3bf4-4ea4-803f-42b458c045ae" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:35:01.124959 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.124890    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "84c23be9-3bf4-4ea4-803f-42b458c045ae" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:35:01.125034 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.125012    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "84c23be9-3bf4-4ea4-803f-42b458c045ae" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:35:01.125306 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.125281    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-kube-api-access-ws5z9" (OuterVolumeSpecName: "kube-api-access-ws5z9") pod "84c23be9-3bf4-4ea4-803f-42b458c045ae" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae"). InnerVolumeSpecName "kube-api-access-ws5z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:35:01.133997 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.133965    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84c23be9-3bf4-4ea4-803f-42b458c045ae-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "84c23be9-3bf4-4ea4-803f-42b458c045ae" (UID: "84c23be9-3bf4-4ea4-803f-42b458c045ae"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:35:01.223521 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.223483    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws5z9\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-kube-api-access-ws5z9\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:35:01.223521 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.223519    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-image-registry-private-configuration\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:35:01.223734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.223537    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-registry-tls\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:35:01.223734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.223549    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84c23be9-3bf4-4ea4-803f-42b458c045ae-bound-sa-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:35:01.223734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.223560    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84c23be9-3bf4-4ea4-803f-42b458c045ae-installation-pull-secrets\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:35:01.223734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.223572    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84c23be9-3bf4-4ea4-803f-42b458c045ae-ca-trust-extracted\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:35:01.223734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.223582    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84c23be9-3bf4-4ea4-803f-42b458c045ae-trusted-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:35:01.751723 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.751687    2569 generic.go:356] "Generic (PLEG): container finished" podID="84c23be9-3bf4-4ea4-803f-42b458c045ae" containerID="a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc" exitCode=0
Sep 10 23:35:01.751949 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.751737    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj"
Sep 10 23:35:01.751949 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.751753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj" event={"ID":"84c23be9-3bf4-4ea4-803f-42b458c045ae","Type":"ContainerDied","Data":"a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc"}
Sep 10 23:35:01.751949 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.751839    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c4d858d-8mjkj" event={"ID":"84c23be9-3bf4-4ea4-803f-42b458c045ae","Type":"ContainerDied","Data":"d540a1f5d7432fd7d0c4d4b47da72716ed399b2e2efd14c55f68f17403353742"}
Sep 10 23:35:01.751949 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.751866    2569 scope.go:117] "RemoveContainer" containerID="a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc"
Sep 10 23:35:01.760184 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.760166    2569 scope.go:117] "RemoveContainer" containerID="a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc"
Sep 10 23:35:01.760482 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:35:01.760457    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc\": container with ID starting with a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc not found: ID does not exist" containerID="a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc"
Sep 10 23:35:01.760570 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.760493    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc"} err="failed to get container status \"a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc\": rpc error: code = NotFound desc = could not find container \"a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc\": container with ID starting with a0c07385f723fea5daae0f9225a38b6d46cbaf96ee494b84359beb1aa58622fc not found: ID does not exist"
Sep 10 23:35:01.773667 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.773643    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56c4d858d-8mjkj"]
Sep 10 23:35:01.777743 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:01.777724    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56c4d858d-8mjkj"]
Sep 10 23:35:02.977685 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:02.977655    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84c23be9-3bf4-4ea4-803f-42b458c045ae" path="/var/lib/kubelet/pods/84c23be9-3bf4-4ea4-803f-42b458c045ae/volumes"
Sep 10 23:35:05.770666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:05.770632    2569 generic.go:356] "Generic (PLEG): container finished" podID="abd2e556-839b-4d11-8c58-566ff67e0fd9" containerID="5917ceddf5f145968abb05702b32a8437aa3df62f5f2d3de9ee27b431c2bffff" exitCode=0
Sep 10 23:35:05.771114 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:05.770695    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc" event={"ID":"abd2e556-839b-4d11-8c58-566ff67e0fd9","Type":"ContainerDied","Data":"5917ceddf5f145968abb05702b32a8437aa3df62f5f2d3de9ee27b431c2bffff"}
Sep 10 23:35:05.771114 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:05.771032    2569 scope.go:117] "RemoveContainer" containerID="5917ceddf5f145968abb05702b32a8437aa3df62f5f2d3de9ee27b431c2bffff"
Sep 10 23:35:06.778333 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:06.777290    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-cfdcc" event={"ID":"abd2e556-839b-4d11-8c58-566ff67e0fd9","Type":"ContainerStarted","Data":"8f2eab05456530e10719af2332fa0900a15120420e158d47f7f59fd7d862f8d5"}
Sep 10 23:35:14.811988 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:14.811930    2569 generic.go:356] "Generic (PLEG): container finished" podID="7f2a3102-f700-476f-a46c-239843bd5356" containerID="f7f84bffde68965c87afe078f5203b6ea37ecfd97110b3d5e030765cf57fc5c2" exitCode=0
Sep 10 23:35:14.812336 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:14.812006    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q" event={"ID":"7f2a3102-f700-476f-a46c-239843bd5356","Type":"ContainerDied","Data":"f7f84bffde68965c87afe078f5203b6ea37ecfd97110b3d5e030765cf57fc5c2"}
Sep 10 23:35:14.812382 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:14.812341    2569 scope.go:117] "RemoveContainer" containerID="f7f84bffde68965c87afe078f5203b6ea37ecfd97110b3d5e030765cf57fc5c2"
Sep 10 23:35:15.817596 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:15.817558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-rbn2q" event={"ID":"7f2a3102-f700-476f-a46c-239843bd5356","Type":"ContainerStarted","Data":"6e754aece7c58bf08349dfa8d7a2dd2cbc9c727b52d421d1e08f4c271c070d7a"}
Sep 10 23:35:15.819255 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:15.819224    2569 generic.go:356] "Generic (PLEG): container finished" podID="a993d388-bd24-4154-b232-2665a25ecab8" containerID="71f989ca205101da059ba48112cf9dd2cda0114fc3bb18936848783ad039d923" exitCode=0
Sep 10 23:35:15.819392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:15.819288    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-vm4gs" event={"ID":"a993d388-bd24-4154-b232-2665a25ecab8","Type":"ContainerDied","Data":"71f989ca205101da059ba48112cf9dd2cda0114fc3bb18936848783ad039d923"}
Sep 10 23:35:15.819631 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:15.819616    2569 scope.go:117] "RemoveContainer" containerID="71f989ca205101da059ba48112cf9dd2cda0114fc3bb18936848783ad039d923"
Sep 10 23:35:16.825692 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:35:16.825658    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-vm4gs" event={"ID":"a993d388-bd24-4154-b232-2665a25ecab8","Type":"ContainerStarted","Data":"4fa19a0a90455b5210dd19e6e7d0e88b73ed8e7906fbe35ac889d28f4f983dcc"}
Sep 10 23:38:06.915707 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:06.915677    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:38:06.920499 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:06.920473    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:38:06.921205 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:06.921187    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:38:06.926950 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:06.926922    2569 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 23:38:06.927663 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:06.927642    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:38:51.684629 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.684589    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"]
Sep 10 23:38:51.685169 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.685073    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84c23be9-3bf4-4ea4-803f-42b458c045ae" containerName="registry"
Sep 10 23:38:51.685169 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.685091    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="84c23be9-3bf4-4ea4-803f-42b458c045ae" containerName="registry"
Sep 10 23:38:51.685294 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.685181    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="84c23be9-3bf4-4ea4-803f-42b458c045ae" containerName="registry"
Sep 10 23:38:51.688200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.688178    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.690878 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.690854    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 23:38:51.691335 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.691312    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 23:38:51.691453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.691440    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 23:38:51.691537 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.691522    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fds9x\""
Sep 10 23:38:51.691589 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.691570    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 23:38:51.713531 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.713508    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"]
Sep 10 23:38:51.826377 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.826341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53514fe5-a017-408c-8251-34237340f3a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.826550 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.826383    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzjlk\" (UniqueName: \"kubernetes.io/projected/53514fe5-a017-408c-8251-34237340f3a5-kube-api-access-lzjlk\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.826550 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.826464    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53514fe5-a017-408c-8251-34237340f3a5-tmp\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.826550 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.826484    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53514fe5-a017-408c-8251-34237340f3a5-manifests\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.826550 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.826511    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53514fe5-a017-408c-8251-34237340f3a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.927308 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.927271    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53514fe5-a017-408c-8251-34237340f3a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.927308 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.927313    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzjlk\" (UniqueName: \"kubernetes.io/projected/53514fe5-a017-408c-8251-34237340f3a5-kube-api-access-lzjlk\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.927549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.927345    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53514fe5-a017-408c-8251-34237340f3a5-tmp\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.927549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.927371    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53514fe5-a017-408c-8251-34237340f3a5-manifests\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.927549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.927408    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53514fe5-a017-408c-8251-34237340f3a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.927822 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.927793    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/53514fe5-a017-408c-8251-34237340f3a5-manifests\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.929782 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.929748    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/53514fe5-a017-408c-8251-34237340f3a5-tmp\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.929926 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.929905    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/53514fe5-a017-408c-8251-34237340f3a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.930004 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.929981    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/53514fe5-a017-408c-8251-34237340f3a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:51.934847 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:51.934803    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzjlk\" (UniqueName: \"kubernetes.io/projected/53514fe5-a017-408c-8251-34237340f3a5-kube-api-access-lzjlk\") pod \"opendatahub-operator-controller-manager-bc8465fcf-2qkrv\" (UID: \"53514fe5-a017-408c-8251-34237340f3a5\") " pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:52.003542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:52.003505    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:52.282811 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:52.282719    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"]
Sep 10 23:38:52.285622 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:38:52.285595    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53514fe5_a017_408c_8251_34237340f3a5.slice/crio-4cc946bfc928f4451b3c0ac8375e67ee5ceb4c5c811c6aaf855b4bf0f2111581 WatchSource:0}: Error finding container 4cc946bfc928f4451b3c0ac8375e67ee5ceb4c5c811c6aaf855b4bf0f2111581: Status 404 returned error can't find the container with id 4cc946bfc928f4451b3c0ac8375e67ee5ceb4c5c811c6aaf855b4bf0f2111581
Sep 10 23:38:52.287552 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:52.287532    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:38:52.653729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:52.653689    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv" event={"ID":"53514fe5-a017-408c-8251-34237340f3a5","Type":"ContainerStarted","Data":"4cc946bfc928f4451b3c0ac8375e67ee5ceb4c5c811c6aaf855b4bf0f2111581"}
Sep 10 23:38:55.670348 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:55.670311    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv" event={"ID":"53514fe5-a017-408c-8251-34237340f3a5","Type":"ContainerStarted","Data":"4656e0ae61b746b728f571c93960ddc88207527b0077dcb35e7b0d76ff7e322d"}
Sep 10 23:38:56.676215 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:56.676181    2569 generic.go:356] "Generic (PLEG): container finished" podID="53514fe5-a017-408c-8251-34237340f3a5" containerID="4656e0ae61b746b728f571c93960ddc88207527b0077dcb35e7b0d76ff7e322d" exitCode=0
Sep 10 23:38:56.676585 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:56.676262    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv" event={"ID":"53514fe5-a017-408c-8251-34237340f3a5","Type":"ContainerDied","Data":"4656e0ae61b746b728f571c93960ddc88207527b0077dcb35e7b0d76ff7e322d"}
Sep 10 23:38:57.683608 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:57.683572    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv" event={"ID":"53514fe5-a017-408c-8251-34237340f3a5","Type":"ContainerStarted","Data":"9da4b5e203ffc04e62ef799c6ff1fa51f031e62ebc267ecdee1ff04f517426ed"}
Sep 10 23:38:57.684079 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:57.683720    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:38:57.705151 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:38:57.705091    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv" podStartSLOduration=2.393964305 podStartE2EDuration="6.705072779s" totalImagesPullingTime="3.228444069s" totalInitContainerRuntime="1.082664405s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:38:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:38:57.703288122 +0000 UTC m=+351.357008295" watchObservedRunningTime="2026-09-10 23:38:57.705072779 +0000 UTC m=+351.358793021"
Sep 10 23:39:08.690336 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:08.690305    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bc8465fcf-2qkrv"
Sep 10 23:39:29.349793 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.349757    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"]
Sep 10 23:39:29.353307 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.353284    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.355896 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.355869    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 23:39:29.356032 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.355869    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rh64g\""
Sep 10 23:39:29.367070 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.367048    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"]
Sep 10 23:39:29.443540 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443493    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.443740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.443740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443581    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.443740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.443740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443710    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.443986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443748    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mk7r\" (UniqueName: \"kubernetes.io/projected/ed4c0472-2c41-4c23-baca-d10f8d8e8094-kube-api-access-9mk7r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.443986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443827    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.443986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443872    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.443986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.443909    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.544559 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.544522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.544559 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.544561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.544783 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.544691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.544783 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.544736    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.544783 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.544753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.544959 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.544801    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.545021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.544956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.545021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.544999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.545118 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.545032    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.545118 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.545067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.545299 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.545276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.545384 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.545368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mk7r\" (UniqueName: \"kubernetes.io/projected/ed4c0472-2c41-4c23-baca-d10f8d8e8094-kube-api-access-9mk7r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.545478 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.545461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.546001 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.545981    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.547084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.547053    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.547430 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.547412    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.556173 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.556146    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed4c0472-2c41-4c23-baca-d10f8d8e8094-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.574650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.574628    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mk7r\" (UniqueName: \"kubernetes.io/projected/ed4c0472-2c41-4c23-baca-d10f8d8e8094-kube-api-access-9mk7r\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh\" (UID: \"ed4c0472-2c41-4c23-baca-d10f8d8e8094\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.666099 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.666067    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:29.946774 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:29.946705    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"]
Sep 10 23:39:29.949800 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:39:29.949769    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4c0472_2c41_4c23_baca_d10f8d8e8094.slice/crio-78c8df213803fd9775eafe0ebfe53974595135f2eaa0f5bf56ea1eaa81201b04 WatchSource:0}: Error finding container 78c8df213803fd9775eafe0ebfe53974595135f2eaa0f5bf56ea1eaa81201b04: Status 404 returned error can't find the container with id 78c8df213803fd9775eafe0ebfe53974595135f2eaa0f5bf56ea1eaa81201b04
Sep 10 23:39:30.826868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:30.826832    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh" event={"ID":"ed4c0472-2c41-4c23-baca-d10f8d8e8094","Type":"ContainerStarted","Data":"78c8df213803fd9775eafe0ebfe53974595135f2eaa0f5bf56ea1eaa81201b04"}
Sep 10 23:39:31.906955 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.906905    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"]
Sep 10 23:39:31.913303 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.913280    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:31.915783 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.915523    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 23:39:31.915892 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.915811    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mtsmn\""
Sep 10 23:39:31.923136 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.922538    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"]
Sep 10 23:39:31.926461 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.926434    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"]
Sep 10 23:39:31.934708 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.934673    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:31.940507 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.939374    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 23:39:31.940507 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.940333    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ljmjr\""
Sep 10 23:39:31.946125 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.945628    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"]
Sep 10 23:39:31.966459 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.966433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdknc\" (UniqueName: \"kubernetes.io/projected/1582b7b7-3f82-4845-9992-46007b14da6b-kube-api-access-fdknc\") pod \"kserve-controller-manager-d6f4b65b8-tf8w2\" (UID: \"1582b7b7-3f82-4845-9992-46007b14da6b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:31.966592 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.966552    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfwb6\" (UniqueName: \"kubernetes.io/projected/eaacc499-1307-4a91-9f42-c17101a52b79-kube-api-access-rfwb6\") pod \"llmisvc-controller-manager-777c6bd7d5-pnll2\" (UID: \"eaacc499-1307-4a91-9f42-c17101a52b79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:31.966592 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.966587    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pnll2\" (UID: \"eaacc499-1307-4a91-9f42-c17101a52b79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:31.966756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:31.966622    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1582b7b7-3f82-4845-9992-46007b14da6b-cert\") pod \"kserve-controller-manager-d6f4b65b8-tf8w2\" (UID: \"1582b7b7-3f82-4845-9992-46007b14da6b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:32.067768 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.067728    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdknc\" (UniqueName: \"kubernetes.io/projected/1582b7b7-3f82-4845-9992-46007b14da6b-kube-api-access-fdknc\") pod \"kserve-controller-manager-d6f4b65b8-tf8w2\" (UID: \"1582b7b7-3f82-4845-9992-46007b14da6b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:32.067966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.067813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfwb6\" (UniqueName: \"kubernetes.io/projected/eaacc499-1307-4a91-9f42-c17101a52b79-kube-api-access-rfwb6\") pod \"llmisvc-controller-manager-777c6bd7d5-pnll2\" (UID: \"eaacc499-1307-4a91-9f42-c17101a52b79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:32.067966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.067848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pnll2\" (UID: \"eaacc499-1307-4a91-9f42-c17101a52b79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:32.067966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.067882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1582b7b7-3f82-4845-9992-46007b14da6b-cert\") pod \"kserve-controller-manager-d6f4b65b8-tf8w2\" (UID: \"1582b7b7-3f82-4845-9992-46007b14da6b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:32.068138 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:39:32.068091    2569 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 10 23:39:32.068190 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:39:32.068158    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1582b7b7-3f82-4845-9992-46007b14da6b-cert podName:1582b7b7-3f82-4845-9992-46007b14da6b nodeName:}" failed. No retries permitted until 2026-09-10 23:39:32.56813468 +0000 UTC m=+386.221854835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1582b7b7-3f82-4845-9992-46007b14da6b-cert") pod "kserve-controller-manager-d6f4b65b8-tf8w2" (UID: "1582b7b7-3f82-4845-9992-46007b14da6b") : secret "kserve-webhook-server-cert" not found
Sep 10 23:39:32.068558 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:39:32.068400    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 10 23:39:32.068558 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:39:32.068463    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert podName:eaacc499-1307-4a91-9f42-c17101a52b79 nodeName:}" failed. No retries permitted until 2026-09-10 23:39:32.568445381 +0000 UTC m=+386.222165550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert") pod "llmisvc-controller-manager-777c6bd7d5-pnll2" (UID: "eaacc499-1307-4a91-9f42-c17101a52b79") : secret "llmisvc-webhook-server-cert" not found
Sep 10 23:39:32.078923 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.078891    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdknc\" (UniqueName: \"kubernetes.io/projected/1582b7b7-3f82-4845-9992-46007b14da6b-kube-api-access-fdknc\") pod \"kserve-controller-manager-d6f4b65b8-tf8w2\" (UID: \"1582b7b7-3f82-4845-9992-46007b14da6b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:32.079143 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.079123    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfwb6\" (UniqueName: \"kubernetes.io/projected/eaacc499-1307-4a91-9f42-c17101a52b79-kube-api-access-rfwb6\") pod \"llmisvc-controller-manager-777c6bd7d5-pnll2\" (UID: \"eaacc499-1307-4a91-9f42-c17101a52b79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:32.572847 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.572803    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pnll2\" (UID: \"eaacc499-1307-4a91-9f42-c17101a52b79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:32.573053 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.572858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1582b7b7-3f82-4845-9992-46007b14da6b-cert\") pod \"kserve-controller-manager-d6f4b65b8-tf8w2\" (UID: \"1582b7b7-3f82-4845-9992-46007b14da6b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:32.573053 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:39:32.572989    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 10 23:39:32.573219 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:39:32.573072    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert podName:eaacc499-1307-4a91-9f42-c17101a52b79 nodeName:}" failed. No retries permitted until 2026-09-10 23:39:33.573047805 +0000 UTC m=+387.226767962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert") pod "llmisvc-controller-manager-777c6bd7d5-pnll2" (UID: "eaacc499-1307-4a91-9f42-c17101a52b79") : secret "llmisvc-webhook-server-cert" not found
Sep 10 23:39:32.575727 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.575703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1582b7b7-3f82-4845-9992-46007b14da6b-cert\") pod \"kserve-controller-manager-d6f4b65b8-tf8w2\" (UID: \"1582b7b7-3f82-4845-9992-46007b14da6b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:32.832437 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.832356    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:32.972252 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.972211    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:39:32.972586 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.972300    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:39:32.972586 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:32.972343    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:39:33.192608 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.192576    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"]
Sep 10 23:39:33.194049 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:39:33.194015    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1582b7b7_3f82_4845_9992_46007b14da6b.slice/crio-b811f9b166c96a886beeb026f07d9559c27ffc1a4de828c636e72fdea416b5e9 WatchSource:0}: Error finding container b811f9b166c96a886beeb026f07d9559c27ffc1a4de828c636e72fdea416b5e9: Status 404 returned error can't find the container with id b811f9b166c96a886beeb026f07d9559c27ffc1a4de828c636e72fdea416b5e9
Sep 10 23:39:33.546623 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.546550    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"]
Sep 10 23:39:33.550135 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.550116    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.552224 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.552205    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 23:39:33.552747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.552733    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 23:39:33.553090 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.553070    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 23:39:33.553248 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.553185    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 23:39:33.553248 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.553210    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 23:39:33.553462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.553447    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h8tbj\""
Sep 10 23:39:33.561982 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.561960    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"]
Sep 10 23:39:33.582800 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.582768    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbed31aa-6127-4291-8136-89624f6996d8-metrics-cert\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.582800 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.582803    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dbed31aa-6127-4291-8136-89624f6996d8-manager-config\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.583042 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.582840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pnll2\" (UID: \"eaacc499-1307-4a91-9f42-c17101a52b79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:33.583042 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.582859    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbed31aa-6127-4291-8136-89624f6996d8-cert\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.583042 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.582883    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmmb9\" (UniqueName: \"kubernetes.io/projected/dbed31aa-6127-4291-8136-89624f6996d8-kube-api-access-dmmb9\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.585705 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.585681    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eaacc499-1307-4a91-9f42-c17101a52b79-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pnll2\" (UID: \"eaacc499-1307-4a91-9f42-c17101a52b79\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:33.684141 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.684109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dbed31aa-6127-4291-8136-89624f6996d8-manager-config\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.684320 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.684157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbed31aa-6127-4291-8136-89624f6996d8-cert\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.684320 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.684184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmmb9\" (UniqueName: \"kubernetes.io/projected/dbed31aa-6127-4291-8136-89624f6996d8-kube-api-access-dmmb9\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.684320 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.684242    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbed31aa-6127-4291-8136-89624f6996d8-metrics-cert\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.684856 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.684830    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dbed31aa-6127-4291-8136-89624f6996d8-manager-config\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.686614 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.686592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbed31aa-6127-4291-8136-89624f6996d8-cert\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.686699 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.686673    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dbed31aa-6127-4291-8136-89624f6996d8-metrics-cert\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.696421 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.696400    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmmb9\" (UniqueName: \"kubernetes.io/projected/dbed31aa-6127-4291-8136-89624f6996d8-kube-api-access-dmmb9\") pod \"lws-controller-manager-bffb6dd48-7zgnf\" (UID: \"dbed31aa-6127-4291-8136-89624f6996d8\") " pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.751507 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.751475    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:33.845367 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.845332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh" event={"ID":"ed4c0472-2c41-4c23-baca-d10f8d8e8094","Type":"ContainerStarted","Data":"b2ad4a0ce5bd424aff874377267862e9e7f982c51dc14a5ea71fa3758c35ba9f"}
Sep 10 23:39:33.847374 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.847310    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2" event={"ID":"1582b7b7-3f82-4845-9992-46007b14da6b","Type":"ContainerStarted","Data":"b811f9b166c96a886beeb026f07d9559c27ffc1a4de828c636e72fdea416b5e9"}
Sep 10 23:39:33.860386 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.860363    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:33.870024 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:33.869949    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh" podStartSLOduration=1.8498488910000002 podStartE2EDuration="4.869917233s" totalImagesPullingTime="3.020068342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:39:33.867830571 +0000 UTC m=+387.521550745" watchObservedRunningTime="2026-09-10 23:39:33.869917233 +0000 UTC m=+387.523637408"
Sep 10 23:39:34.034190 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:34.034161    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"]
Sep 10 23:39:34.036709 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:39:34.036675    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeaacc499_1307_4a91_9f42_c17101a52b79.slice/crio-62f1562a0c7380941e8b966c65d3afd1ddc38b9a4279b675950e9ad53f4af438 WatchSource:0}: Error finding container 62f1562a0c7380941e8b966c65d3afd1ddc38b9a4279b675950e9ad53f4af438: Status 404 returned error can't find the container with id 62f1562a0c7380941e8b966c65d3afd1ddc38b9a4279b675950e9ad53f4af438
Sep 10 23:39:34.151555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:34.151519    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"]
Sep 10 23:39:34.155660 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:39:34.155624    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbed31aa_6127_4291_8136_89624f6996d8.slice/crio-94dccf714ebb58fe5e1b7980f4f85523871df96d3805c0645fa266740114237f WatchSource:0}: Error finding container 94dccf714ebb58fe5e1b7980f4f85523871df96d3805c0645fa266740114237f: Status 404 returned error can't find the container with id 94dccf714ebb58fe5e1b7980f4f85523871df96d3805c0645fa266740114237f
Sep 10 23:39:34.666588 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:34.666553    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:34.672648 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:34.672539    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:34.853643 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:34.853605    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2" event={"ID":"eaacc499-1307-4a91-9f42-c17101a52b79","Type":"ContainerStarted","Data":"62f1562a0c7380941e8b966c65d3afd1ddc38b9a4279b675950e9ad53f4af438"}
Sep 10 23:39:34.854987 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:34.854961    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf" event={"ID":"dbed31aa-6127-4291-8136-89624f6996d8","Type":"ContainerStarted","Data":"94dccf714ebb58fe5e1b7980f4f85523871df96d3805c0645fa266740114237f"}
Sep 10 23:39:34.855179 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:34.855161    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:34.856368 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:34.856349    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh"
Sep 10 23:39:36.868698 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:36.868659    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2" event={"ID":"1582b7b7-3f82-4845-9992-46007b14da6b","Type":"ContainerStarted","Data":"01c8430ca54ac142b0c51b3d68a355a2184784fc6c7e1547c07c18d70afe56d6"}
Sep 10 23:39:36.869200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:36.868804    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:39:36.889051 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:36.888989    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2" podStartSLOduration=3.228384682 podStartE2EDuration="5.888966946s" totalImagesPullingTime="2.660582264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:39:36.886433639 +0000 UTC m=+390.540153815" watchObservedRunningTime="2026-09-10 23:39:36.888966946 +0000 UTC m=+390.542687120"
Sep 10 23:39:38.881025 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:38.880985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf" event={"ID":"dbed31aa-6127-4291-8136-89624f6996d8","Type":"ContainerStarted","Data":"f2048847eebeaae6b92a649117b2f026fd0ede7d955f47830fc89989794c80ba"}
Sep 10 23:39:38.881514 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:38.881064    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:38.882613 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:38.882588    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2" event={"ID":"eaacc499-1307-4a91-9f42-c17101a52b79","Type":"ContainerStarted","Data":"fef4ca9020490608d583f4d1472d8fcae59b06d348c75d414d2f3e1aa16ce5a1"}
Sep 10 23:39:38.882725 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:38.882705    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:39:38.899281 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:38.899236    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf" podStartSLOduration=1.5340025659999998 podStartE2EDuration="5.899221341s" totalImagesPullingTime="4.365218775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:39:38.897893933 +0000 UTC m=+392.551614103" watchObservedRunningTime="2026-09-10 23:39:38.899221341 +0000 UTC m=+392.552941513"
Sep 10 23:39:49.889484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:49.889454    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bffb6dd48-7zgnf"
Sep 10 23:39:49.907619 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:39:49.907566    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2" podStartSLOduration=14.383444273 podStartE2EDuration="18.907547992s" totalImagesPullingTime="4.524103719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:39:38.913687605 +0000 UTC m=+392.567407778" watchObservedRunningTime="2026-09-10 23:39:49.907547992 +0000 UTC m=+403.561268212"
Sep 10 23:40:07.880342 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:40:07.880311    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-tf8w2"
Sep 10 23:40:09.889802 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:40:09.889766    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pnll2"
Sep 10 23:41:02.487237 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.487193    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"]
Sep 10 23:41:02.490555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.490535    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:02.493030 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.493005    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8v52d\""
Sep 10 23:41:02.493030 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.493022    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 23:41:02.493207 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.493023    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 23:41:02.506068 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.506041    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"]
Sep 10 23:41:02.660089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.660036    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e4d058-1c34-432e-909a-a557d514b3c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" (UID: \"20e4d058-1c34-432e-909a-a557d514b3c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:02.660269 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.660151    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8dh7\" (UniqueName: \"kubernetes.io/projected/20e4d058-1c34-432e-909a-a557d514b3c7-kube-api-access-s8dh7\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" (UID: \"20e4d058-1c34-432e-909a-a557d514b3c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:02.761640 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.761536    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e4d058-1c34-432e-909a-a557d514b3c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" (UID: \"20e4d058-1c34-432e-909a-a557d514b3c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:02.761640 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.761625    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8dh7\" (UniqueName: \"kubernetes.io/projected/20e4d058-1c34-432e-909a-a557d514b3c7-kube-api-access-s8dh7\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" (UID: \"20e4d058-1c34-432e-909a-a557d514b3c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:02.761930 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.761909    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e4d058-1c34-432e-909a-a557d514b3c7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" (UID: \"20e4d058-1c34-432e-909a-a557d514b3c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:02.772637 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.772600    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8dh7\" (UniqueName: \"kubernetes.io/projected/20e4d058-1c34-432e-909a-a557d514b3c7-kube-api-access-s8dh7\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" (UID: \"20e4d058-1c34-432e-909a-a557d514b3c7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:02.800591 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:02.800558    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:03.077906 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.077722    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"]
Sep 10 23:41:03.081052 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:41:03.081026    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20e4d058_1c34_432e_909a_a557d514b3c7.slice/crio-a03b980f467a724281ac87840221f80343bec67da54dcdaecff5af211f8417f1 WatchSource:0}: Error finding container a03b980f467a724281ac87840221f80343bec67da54dcdaecff5af211f8417f1: Status 404 returned error can't find the container with id a03b980f467a724281ac87840221f80343bec67da54dcdaecff5af211f8417f1
Sep 10 23:41:03.249583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.249551    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg" event={"ID":"20e4d058-1c34-432e-909a-a557d514b3c7","Type":"ContainerStarted","Data":"a03b980f467a724281ac87840221f80343bec67da54dcdaecff5af211f8417f1"}
Sep 10 23:41:03.467313 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.467281    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"]
Sep 10 23:41:03.476723 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.476694    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"]
Sep 10 23:41:03.489899 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.489867    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"]
Sep 10 23:41:03.494466 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.494445    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:03.508904 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.508869    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"]
Sep 10 23:41:03.569542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.569506    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-72krz\" (UID: \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:03.569542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.569541    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtf57\" (UniqueName: \"kubernetes.io/projected/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-kube-api-access-gtf57\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-72krz\" (UID: \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:03.670306 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.670277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-72krz\" (UID: \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:03.670468 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.670312    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtf57\" (UniqueName: \"kubernetes.io/projected/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-kube-api-access-gtf57\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-72krz\" (UID: \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:03.670652 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.670632    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-72krz\" (UID: \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:03.677763 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.677736    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtf57\" (UniqueName: \"kubernetes.io/projected/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-kube-api-access-gtf57\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-72krz\" (UID: \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:03.805707 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:03.805628    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:04.099017 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:04.098983    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"]
Sep 10 23:41:04.102954 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:41:04.102883    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b6947e_3ad1_489c_9a84_ff05b49c4eae.slice/crio-9ec0b87d0f7cb1caf393fcb453f372515ffa7e318c25ce876c253b7bc90d4570 WatchSource:0}: Error finding container 9ec0b87d0f7cb1caf393fcb453f372515ffa7e318c25ce876c253b7bc90d4570: Status 404 returned error can't find the container with id 9ec0b87d0f7cb1caf393fcb453f372515ffa7e318c25ce876c253b7bc90d4570
Sep 10 23:41:04.258013 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:04.257972    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz" event={"ID":"f4b6947e-3ad1-489c-9a84-ff05b49c4eae","Type":"ContainerStarted","Data":"9ec0b87d0f7cb1caf393fcb453f372515ffa7e318c25ce876c253b7bc90d4570"}
Sep 10 23:41:08.284089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.284049    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz" event={"ID":"f4b6947e-3ad1-489c-9a84-ff05b49c4eae","Type":"ContainerStarted","Data":"658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26"}
Sep 10 23:41:08.284536 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.284144    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:08.285894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.285868    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" containerName="manager" containerID="cri-o://f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f" gracePeriod=2
Sep 10 23:41:08.306713 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.306658    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz" podStartSLOduration=1.818559358 podStartE2EDuration="5.306640302s" totalImagesPullingTime="3.488080944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:41:08.30519933 +0000 UTC m=+481.958919516" watchObservedRunningTime="2026-09-10 23:41:08.306640302 +0000 UTC m=+481.960360474"
Sep 10 23:41:08.307079 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:41:08.307051    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:08.530090 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.530065    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:08.532101 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:41:08.532069    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:08.613834 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.613808    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e4d058-1c34-432e-909a-a557d514b3c7-extensions-socket-volume\") pod \"20e4d058-1c34-432e-909a-a557d514b3c7\" (UID: \"20e4d058-1c34-432e-909a-a557d514b3c7\") "
Sep 10 23:41:08.614084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.613840    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8dh7\" (UniqueName: \"kubernetes.io/projected/20e4d058-1c34-432e-909a-a557d514b3c7-kube-api-access-s8dh7\") pod \"20e4d058-1c34-432e-909a-a557d514b3c7\" (UID: \"20e4d058-1c34-432e-909a-a557d514b3c7\") "
Sep 10 23:41:08.614192 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.614164    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e4d058-1c34-432e-909a-a557d514b3c7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "20e4d058-1c34-432e-909a-a557d514b3c7" (UID: "20e4d058-1c34-432e-909a-a557d514b3c7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:41:08.616083 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.616053    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e4d058-1c34-432e-909a-a557d514b3c7-kube-api-access-s8dh7" (OuterVolumeSpecName: "kube-api-access-s8dh7") pod "20e4d058-1c34-432e-909a-a557d514b3c7" (UID: "20e4d058-1c34-432e-909a-a557d514b3c7"). InnerVolumeSpecName "kube-api-access-s8dh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:41:08.715314 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.715281    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/20e4d058-1c34-432e-909a-a557d514b3c7-extensions-socket-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:41:08.715314 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.715309    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8dh7\" (UniqueName: \"kubernetes.io/projected/20e4d058-1c34-432e-909a-a557d514b3c7-kube-api-access-s8dh7\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:41:08.978792 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:08.978717    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" path="/var/lib/kubelet/pods/20e4d058-1c34-432e-909a-a557d514b3c7/volumes"
Sep 10 23:41:09.292224 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:09.292130    2569 generic.go:356] "Generic (PLEG): container finished" podID="20e4d058-1c34-432e-909a-a557d514b3c7" containerID="f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f" exitCode=2
Sep 10 23:41:09.292224 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:09.292179    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:09.292682 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:09.292226    2569 scope.go:117] "RemoveContainer" containerID="f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f"
Sep 10 23:41:09.294223 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:41:09.294195    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:09.296082 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:41:09.296060    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-kjjfg\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-kjjfg"
Sep 10 23:41:09.300416 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:09.300400    2569 scope.go:117] "RemoveContainer" containerID="f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f"
Sep 10 23:41:09.300636 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:41:09.300619    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f\": container with ID starting with f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f not found: ID does not exist" containerID="f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f"
Sep 10 23:41:09.300688 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:09.300644    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f"} err="failed to get container status \"f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f\": rpc error: code = NotFound desc = could not find container \"f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f\": container with ID starting with f1020894dfa329b536d08a4634bcb1bc6923df01b2568b8b5d57c973f1b8679f not found: ID does not exist"
Sep 10 23:41:19.295004 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:19.294973    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:31.637689 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:31.637644    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"]
Sep 10 23:41:31.638215 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:31.637960    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz" podUID="f4b6947e-3ad1-489c-9a84-ff05b49c4eae" containerName="manager" containerID="cri-o://658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26" gracePeriod=10
Sep 10 23:41:31.894920 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:31.894864    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:32.004818 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.004780    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-extensions-socket-volume\") pod \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\" (UID: \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\") "
Sep 10 23:41:32.005042 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.004833    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtf57\" (UniqueName: \"kubernetes.io/projected/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-kube-api-access-gtf57\") pod \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\" (UID: \"f4b6947e-3ad1-489c-9a84-ff05b49c4eae\") "
Sep 10 23:41:32.005425 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.005396    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f4b6947e-3ad1-489c-9a84-ff05b49c4eae" (UID: "f4b6947e-3ad1-489c-9a84-ff05b49c4eae"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:41:32.006980 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.006958    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-kube-api-access-gtf57" (OuterVolumeSpecName: "kube-api-access-gtf57") pod "f4b6947e-3ad1-489c-9a84-ff05b49c4eae" (UID: "f4b6947e-3ad1-489c-9a84-ff05b49c4eae"). InnerVolumeSpecName "kube-api-access-gtf57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:41:32.106035 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.105999    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-extensions-socket-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:41:32.106035 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.106024    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtf57\" (UniqueName: \"kubernetes.io/projected/f4b6947e-3ad1-489c-9a84-ff05b49c4eae-kube-api-access-gtf57\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:41:32.400650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.400614    2569 generic.go:356] "Generic (PLEG): container finished" podID="f4b6947e-3ad1-489c-9a84-ff05b49c4eae" containerID="658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26" exitCode=0
Sep 10 23:41:32.400843 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.400658    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"
Sep 10 23:41:32.400843 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.400703    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz" event={"ID":"f4b6947e-3ad1-489c-9a84-ff05b49c4eae","Type":"ContainerDied","Data":"658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26"}
Sep 10 23:41:32.400843 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.400758    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz" event={"ID":"f4b6947e-3ad1-489c-9a84-ff05b49c4eae","Type":"ContainerDied","Data":"9ec0b87d0f7cb1caf393fcb453f372515ffa7e318c25ce876c253b7bc90d4570"}
Sep 10 23:41:32.400843 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.400787    2569 scope.go:117] "RemoveContainer" containerID="658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26"
Sep 10 23:41:32.409740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.409721    2569 scope.go:117] "RemoveContainer" containerID="658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26"
Sep 10 23:41:32.410029 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:41:32.410008    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26\": container with ID starting with 658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26 not found: ID does not exist" containerID="658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26"
Sep 10 23:41:32.410102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.410038    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26"} err="failed to get container status \"658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26\": rpc error: code = NotFound desc = could not find container \"658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26\": container with ID starting with 658757ee3ae36912058f3f26a858c87f1342670ec5b9f2d4040b1a6e90ecba26 not found: ID does not exist"
Sep 10 23:41:32.423414 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.423389    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"]
Sep 10 23:41:32.430231 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.430208    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-72krz"]
Sep 10 23:41:32.978654 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:32.978624    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4b6947e-3ad1-489c-9a84-ff05b49c4eae" path="/var/lib/kubelet/pods/f4b6947e-3ad1-489c-9a84-ff05b49c4eae/volumes"
Sep 10 23:41:35.634382 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.634348    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"]
Sep 10 23:41:35.634764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.634668    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4b6947e-3ad1-489c-9a84-ff05b49c4eae" containerName="manager"
Sep 10 23:41:35.634764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.634678    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b6947e-3ad1-489c-9a84-ff05b49c4eae" containerName="manager"
Sep 10 23:41:35.634764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.634698    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" containerName="manager"
Sep 10 23:41:35.634764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.634703    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" containerName="manager"
Sep 10 23:41:35.634764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.634755    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e4d058-1c34-432e-909a-a557d514b3c7" containerName="manager"
Sep 10 23:41:35.634764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.634762    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4b6947e-3ad1-489c-9a84-ff05b49c4eae" containerName="manager"
Sep 10 23:41:35.639048 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.639024    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.641223 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.641195    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dm9k2\""
Sep 10 23:41:35.648914 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.648888    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"]
Sep 10 23:41:35.739149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739117    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.739149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739150    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.739407 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvz48\" (UniqueName: \"kubernetes.io/projected/1ddd5db5-9ef6-4dea-a625-36023eb3e937-kube-api-access-qvz48\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.739407 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.739407 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.739407 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739369    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.739565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739425    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.739565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739447    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.739565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.739465    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.839991    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840042    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840069    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840091    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840127    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840148    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840183    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvz48\" (UniqueName: \"kubernetes.io/projected/1ddd5db5-9ef6-4dea-a625-36023eb3e937-kube-api-access-qvz48\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840261    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840465    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840526    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840748 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840697    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840806 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840754    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.840979 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.840927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.842520 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.842499    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.842717 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.842703    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.848273 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.848252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ddd5db5-9ef6-4dea-a625-36023eb3e937-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.848466 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.848449    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvz48\" (UniqueName: \"kubernetes.io/projected/1ddd5db5-9ef6-4dea-a625-36023eb3e937-kube-api-access-qvz48\") pod \"maas-default-gateway-openshift-default-78bb78ff78-n8b68\" (UID: \"1ddd5db5-9ef6-4dea-a625-36023eb3e937\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:35.950630 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:35.950546    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:36.230632 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:36.230602    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"]
Sep 10 23:41:36.232606 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:41:36.232576    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ddd5db5_9ef6_4dea_a625_36023eb3e937.slice/crio-b32ab8d41a3222095cbfe3d37644ff6fd1f07df7efa90dd6fb7b8f902c1aba51 WatchSource:0}: Error finding container b32ab8d41a3222095cbfe3d37644ff6fd1f07df7efa90dd6fb7b8f902c1aba51: Status 404 returned error can't find the container with id b32ab8d41a3222095cbfe3d37644ff6fd1f07df7efa90dd6fb7b8f902c1aba51
Sep 10 23:41:36.234792 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:36.234751    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:41:36.234870 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:36.234849    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:41:36.234915 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:36.234877    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:41:36.424908 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:36.424872    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" event={"ID":"1ddd5db5-9ef6-4dea-a625-36023eb3e937","Type":"ContainerStarted","Data":"3c43aa72679e666681aa4bf4e95e645ac4a19515563dd4d6816735e6bcf1f9ff"}
Sep 10 23:41:36.425136 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:36.424999    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" event={"ID":"1ddd5db5-9ef6-4dea-a625-36023eb3e937","Type":"ContainerStarted","Data":"b32ab8d41a3222095cbfe3d37644ff6fd1f07df7efa90dd6fb7b8f902c1aba51"}
Sep 10 23:41:36.443536 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:36.443477    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" podStartSLOduration=1.44345648 podStartE2EDuration="1.44345648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:41:36.443105739 +0000 UTC m=+510.096825913" watchObservedRunningTime="2026-09-10 23:41:36.44345648 +0000 UTC m=+510.097176654"
Sep 10 23:41:36.950844 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:36.950806    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:37.955838 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:37.955808    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:38.435634 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:38.435595    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:38.436613 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:38.436588    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:41:49.698249 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.698212    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-4sr2t"]
Sep 10 23:41:49.701757 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.701741    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-4sr2t"
Sep 10 23:41:49.703736 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.703706    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 23:41:49.703860 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.703779    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 23:41:49.704284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.704268    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bn728\""
Sep 10 23:41:49.707922 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.707899    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4sr2t"]
Sep 10 23:41:49.862843 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.862809    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/6f35cfe0-96eb-4b73-84d1-386930a8b26d-kube-api-access-db4cx\") pod \"authorino-59c974bd48-4sr2t\" (UID: \"6f35cfe0-96eb-4b73-84d1-386930a8b26d\") " pod="kuadrant-system/authorino-59c974bd48-4sr2t"
Sep 10 23:41:49.963616 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.963529    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/6f35cfe0-96eb-4b73-84d1-386930a8b26d-kube-api-access-db4cx\") pod \"authorino-59c974bd48-4sr2t\" (UID: \"6f35cfe0-96eb-4b73-84d1-386930a8b26d\") " pod="kuadrant-system/authorino-59c974bd48-4sr2t"
Sep 10 23:41:49.971139 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:49.971109    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/6f35cfe0-96eb-4b73-84d1-386930a8b26d-kube-api-access-db4cx\") pod \"authorino-59c974bd48-4sr2t\" (UID: \"6f35cfe0-96eb-4b73-84d1-386930a8b26d\") " pod="kuadrant-system/authorino-59c974bd48-4sr2t"
Sep 10 23:41:50.012563 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:50.012522    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-4sr2t"
Sep 10 23:41:50.279064 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:50.279031    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4sr2t"]
Sep 10 23:41:50.281533 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:41:50.281505    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f35cfe0_96eb_4b73_84d1_386930a8b26d.slice/crio-4d333836683530ce667a09ad06e75f4639401db3bd965b603e430dcd1307c719 WatchSource:0}: Error finding container 4d333836683530ce667a09ad06e75f4639401db3bd965b603e430dcd1307c719: Status 404 returned error can't find the container with id 4d333836683530ce667a09ad06e75f4639401db3bd965b603e430dcd1307c719
Sep 10 23:41:50.489516 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:50.489484    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-4sr2t" event={"ID":"6f35cfe0-96eb-4b73-84d1-386930a8b26d","Type":"ContainerStarted","Data":"4d333836683530ce667a09ad06e75f4639401db3bd965b603e430dcd1307c719"}
Sep 10 23:41:52.501065 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:52.501028    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-4sr2t" event={"ID":"6f35cfe0-96eb-4b73-84d1-386930a8b26d","Type":"ContainerStarted","Data":"db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906"}
Sep 10 23:41:52.513881 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:41:52.513804    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-4sr2t" podStartSLOduration=2.041001527 podStartE2EDuration="3.513788274s" totalImagesPullingTime="1.472786747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:41:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:41:52.513510065 +0000 UTC m=+526.167230237" watchObservedRunningTime="2026-09-10 23:41:52.513788274 +0000 UTC m=+526.167508429"
Sep 10 23:42:21.406532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.406498    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-r7ht2"]
Sep 10 23:42:21.411332 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.411310    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-r7ht2"
Sep 10 23:42:21.417480 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.417453    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-r7ht2"]
Sep 10 23:42:21.431506 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.431477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2ph2\" (UniqueName: \"kubernetes.io/projected/c05482b9-7d8f-4651-b251-f5cc9aef14fc-kube-api-access-v2ph2\") pod \"authorino-557f876f69-r7ht2\" (UID: \"c05482b9-7d8f-4651-b251-f5cc9aef14fc\") " pod="kuadrant-system/authorino-557f876f69-r7ht2"
Sep 10 23:42:21.497213 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.497178    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-r7ht2"]
Sep 10 23:42:21.497439 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:42:21.497420    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-v2ph2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-r7ht2" podUID="c05482b9-7d8f-4651-b251-f5cc9aef14fc"
Sep 10 23:42:21.521636 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.521602    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qzgrk"]
Sep 10 23:42:21.525011 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.524988    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.526863 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.526844    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 23:42:21.532276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.532211    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2ph2\" (UniqueName: \"kubernetes.io/projected/c05482b9-7d8f-4651-b251-f5cc9aef14fc-kube-api-access-v2ph2\") pod \"authorino-557f876f69-r7ht2\" (UID: \"c05482b9-7d8f-4651-b251-f5cc9aef14fc\") " pod="kuadrant-system/authorino-557f876f69-r7ht2"
Sep 10 23:42:21.532653 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.532630    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qzgrk"]
Sep 10 23:42:21.539160 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.539137    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2ph2\" (UniqueName: \"kubernetes.io/projected/c05482b9-7d8f-4651-b251-f5cc9aef14fc-kube-api-access-v2ph2\") pod \"authorino-557f876f69-r7ht2\" (UID: \"c05482b9-7d8f-4651-b251-f5cc9aef14fc\") " pod="kuadrant-system/authorino-557f876f69-r7ht2"
Sep 10 23:42:21.625302 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.625257    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qzgrk"]
Sep 10 23:42:21.625593 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:42:21.625567    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-t4qlq tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-qzgrk" podUID="bc717724-8d29-4492-bec4-15d383f04f13"
Sep 10 23:42:21.632282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.632254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-r7ht2"
Sep 10 23:42:21.632410 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.632254    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.632674 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.632655    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc717724-8d29-4492-bec4-15d383f04f13-tls-cert\") pod \"authorino-6d4dfc649f-qzgrk\" (UID: \"bc717724-8d29-4492-bec4-15d383f04f13\") " pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.632752 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.632727    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4qlq\" (UniqueName: \"kubernetes.io/projected/bc717724-8d29-4492-bec4-15d383f04f13-kube-api-access-t4qlq\") pod \"authorino-6d4dfc649f-qzgrk\" (UID: \"bc717724-8d29-4492-bec4-15d383f04f13\") " pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.639612 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.639582    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-r7ht2"
Sep 10 23:42:21.642656 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.642637    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.650664 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.650642    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-594d848786-h72mj"]
Sep 10 23:42:21.653789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.653772    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:42:21.660465 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.660440    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-594d848786-h72mj"]
Sep 10 23:42:21.733469 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.733433    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2ph2\" (UniqueName: \"kubernetes.io/projected/c05482b9-7d8f-4651-b251-f5cc9aef14fc-kube-api-access-v2ph2\") pod \"c05482b9-7d8f-4651-b251-f5cc9aef14fc\" (UID: \"c05482b9-7d8f-4651-b251-f5cc9aef14fc\") "
Sep 10 23:42:21.733647 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.733570    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc717724-8d29-4492-bec4-15d383f04f13-tls-cert\") pod \"authorino-6d4dfc649f-qzgrk\" (UID: \"bc717724-8d29-4492-bec4-15d383f04f13\") " pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.733693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.733649    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9968035a-b5fe-498b-8888-150bd2305c7a-tls-cert\") pod \"authorino-594d848786-h72mj\" (UID: \"9968035a-b5fe-498b-8888-150bd2305c7a\") " pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:42:21.733693 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.733681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slkgq\" (UniqueName: \"kubernetes.io/projected/9968035a-b5fe-498b-8888-150bd2305c7a-kube-api-access-slkgq\") pod \"authorino-594d848786-h72mj\" (UID: \"9968035a-b5fe-498b-8888-150bd2305c7a\") " pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:42:21.733784 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.733709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4qlq\" (UniqueName: \"kubernetes.io/projected/bc717724-8d29-4492-bec4-15d383f04f13-kube-api-access-t4qlq\") pod \"authorino-6d4dfc649f-qzgrk\" (UID: \"bc717724-8d29-4492-bec4-15d383f04f13\") " pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.735647 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.735611    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c05482b9-7d8f-4651-b251-f5cc9aef14fc-kube-api-access-v2ph2" (OuterVolumeSpecName: "kube-api-access-v2ph2") pod "c05482b9-7d8f-4651-b251-f5cc9aef14fc" (UID: "c05482b9-7d8f-4651-b251-f5cc9aef14fc"). InnerVolumeSpecName "kube-api-access-v2ph2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:42:21.736118 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.736104    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc717724-8d29-4492-bec4-15d383f04f13-tls-cert\") pod \"authorino-6d4dfc649f-qzgrk\" (UID: \"bc717724-8d29-4492-bec4-15d383f04f13\") " pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.740311 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.740286    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4qlq\" (UniqueName: \"kubernetes.io/projected/bc717724-8d29-4492-bec4-15d383f04f13-kube-api-access-t4qlq\") pod \"authorino-6d4dfc649f-qzgrk\" (UID: \"bc717724-8d29-4492-bec4-15d383f04f13\") " pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:21.834643 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.834610    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc717724-8d29-4492-bec4-15d383f04f13-tls-cert\") pod \"bc717724-8d29-4492-bec4-15d383f04f13\" (UID: \"bc717724-8d29-4492-bec4-15d383f04f13\") "
Sep 10 23:42:21.834643 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.834645    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4qlq\" (UniqueName: \"kubernetes.io/projected/bc717724-8d29-4492-bec4-15d383f04f13-kube-api-access-t4qlq\") pod \"bc717724-8d29-4492-bec4-15d383f04f13\" (UID: \"bc717724-8d29-4492-bec4-15d383f04f13\") "
Sep 10 23:42:21.834822 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.834753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9968035a-b5fe-498b-8888-150bd2305c7a-tls-cert\") pod \"authorino-594d848786-h72mj\" (UID: \"9968035a-b5fe-498b-8888-150bd2305c7a\") " pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:42:21.834958 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.834911    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slkgq\" (UniqueName: \"kubernetes.io/projected/9968035a-b5fe-498b-8888-150bd2305c7a-kube-api-access-slkgq\") pod \"authorino-594d848786-h72mj\" (UID: \"9968035a-b5fe-498b-8888-150bd2305c7a\") " pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:42:21.835074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.835057    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2ph2\" (UniqueName: \"kubernetes.io/projected/c05482b9-7d8f-4651-b251-f5cc9aef14fc-kube-api-access-v2ph2\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:42:21.836853 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.836824    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc717724-8d29-4492-bec4-15d383f04f13-kube-api-access-t4qlq" (OuterVolumeSpecName: "kube-api-access-t4qlq") pod "bc717724-8d29-4492-bec4-15d383f04f13" (UID: "bc717724-8d29-4492-bec4-15d383f04f13"). InnerVolumeSpecName "kube-api-access-t4qlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:42:21.837009 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.836850    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc717724-8d29-4492-bec4-15d383f04f13-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "bc717724-8d29-4492-bec4-15d383f04f13" (UID: "bc717724-8d29-4492-bec4-15d383f04f13"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:42:21.837268 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.837249    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9968035a-b5fe-498b-8888-150bd2305c7a-tls-cert\") pod \"authorino-594d848786-h72mj\" (UID: \"9968035a-b5fe-498b-8888-150bd2305c7a\") " pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:42:21.842629 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.842607    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slkgq\" (UniqueName: \"kubernetes.io/projected/9968035a-b5fe-498b-8888-150bd2305c7a-kube-api-access-slkgq\") pod \"authorino-594d848786-h72mj\" (UID: \"9968035a-b5fe-498b-8888-150bd2305c7a\") " pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:42:21.936045 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.935963    2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bc717724-8d29-4492-bec4-15d383f04f13-tls-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:42:21.936045 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.935992    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4qlq\" (UniqueName: \"kubernetes.io/projected/bc717724-8d29-4492-bec4-15d383f04f13-kube-api-access-t4qlq\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:42:21.963509 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:21.963478    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:42:22.231009 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.230916    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-594d848786-h72mj"]
Sep 10 23:42:22.234525 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:42:22.234494    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9968035a_b5fe_498b_8888_150bd2305c7a.slice/crio-fdda4a694e681d29d1daec7ab912973a60dbbb7003e171384deb17588326d492 WatchSource:0}: Error finding container fdda4a694e681d29d1daec7ab912973a60dbbb7003e171384deb17588326d492: Status 404 returned error can't find the container with id fdda4a694e681d29d1daec7ab912973a60dbbb7003e171384deb17588326d492
Sep 10 23:42:22.637971 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.637916    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594d848786-h72mj" event={"ID":"9968035a-b5fe-498b-8888-150bd2305c7a","Type":"ContainerStarted","Data":"fdda4a694e681d29d1daec7ab912973a60dbbb7003e171384deb17588326d492"}
Sep 10 23:42:22.637971 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.637963    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-qzgrk"
Sep 10 23:42:22.638392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.637945    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-r7ht2"
Sep 10 23:42:22.667320 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.667297    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-r7ht2"]
Sep 10 23:42:22.669838 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.669810    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-r7ht2"]
Sep 10 23:42:22.689521 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.689481    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qzgrk"]
Sep 10 23:42:22.692262 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.692239    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-qzgrk"]
Sep 10 23:42:22.978097 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.978021    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc717724-8d29-4492-bec4-15d383f04f13" path="/var/lib/kubelet/pods/bc717724-8d29-4492-bec4-15d383f04f13/volumes"
Sep 10 23:42:22.978262 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:22.978250    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c05482b9-7d8f-4651-b251-f5cc9aef14fc" path="/var/lib/kubelet/pods/c05482b9-7d8f-4651-b251-f5cc9aef14fc/volumes"
Sep 10 23:42:23.644217 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:23.644180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594d848786-h72mj" event={"ID":"9968035a-b5fe-498b-8888-150bd2305c7a","Type":"ContainerStarted","Data":"b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb"}
Sep 10 23:42:23.659352 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:23.659306    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-594d848786-h72mj" podStartSLOduration=2.246874513 podStartE2EDuration="2.659291814s" totalImagesPullingTime="412.417301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:42:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:42:23.657246852 +0000 UTC m=+557.310967026" watchObservedRunningTime="2026-09-10 23:42:23.659291814 +0000 UTC m=+557.313012000"
Sep 10 23:42:23.682660 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:23.682627    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4sr2t"]
Sep 10 23:42:23.682813 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:23.682793    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-4sr2t" podUID="6f35cfe0-96eb-4b73-84d1-386930a8b26d" containerName="authorino" containerID="cri-o://db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906" gracePeriod=30
Sep 10 23:42:23.929781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:23.929757    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-4sr2t"
Sep 10 23:42:24.052244 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.052205    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/6f35cfe0-96eb-4b73-84d1-386930a8b26d-kube-api-access-db4cx\") pod \"6f35cfe0-96eb-4b73-84d1-386930a8b26d\" (UID: \"6f35cfe0-96eb-4b73-84d1-386930a8b26d\") "
Sep 10 23:42:24.054338 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.054312    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f35cfe0-96eb-4b73-84d1-386930a8b26d-kube-api-access-db4cx" (OuterVolumeSpecName: "kube-api-access-db4cx") pod "6f35cfe0-96eb-4b73-84d1-386930a8b26d" (UID: "6f35cfe0-96eb-4b73-84d1-386930a8b26d"). InnerVolumeSpecName "kube-api-access-db4cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:42:24.153096 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.153059    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db4cx\" (UniqueName: \"kubernetes.io/projected/6f35cfe0-96eb-4b73-84d1-386930a8b26d-kube-api-access-db4cx\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:42:24.650220 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.650186    2569 generic.go:356] "Generic (PLEG): container finished" podID="6f35cfe0-96eb-4b73-84d1-386930a8b26d" containerID="db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906" exitCode=0
Sep 10 23:42:24.650632 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.650240    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-4sr2t"
Sep 10 23:42:24.650632 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.650272    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-4sr2t" event={"ID":"6f35cfe0-96eb-4b73-84d1-386930a8b26d","Type":"ContainerDied","Data":"db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906"}
Sep 10 23:42:24.650632 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.650314    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-4sr2t" event={"ID":"6f35cfe0-96eb-4b73-84d1-386930a8b26d","Type":"ContainerDied","Data":"4d333836683530ce667a09ad06e75f4639401db3bd965b603e430dcd1307c719"}
Sep 10 23:42:24.650632 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.650331    2569 scope.go:117] "RemoveContainer" containerID="db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906"
Sep 10 23:42:24.659138 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.659120    2569 scope.go:117] "RemoveContainer" containerID="db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906"
Sep 10 23:42:24.659397 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:42:24.659379    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906\": container with ID starting with db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906 not found: ID does not exist" containerID="db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906"
Sep 10 23:42:24.659447 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.659406    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906"} err="failed to get container status \"db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906\": rpc error: code = NotFound desc = could not find container \"db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906\": container with ID starting with db057b5efa4b36da4d308a67c8dd133580d56bb728e604c347e03cbf72dfd906 not found: ID does not exist"
Sep 10 23:42:24.668981 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.668954    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4sr2t"]
Sep 10 23:42:24.671062 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.671043    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-4sr2t"]
Sep 10 23:42:24.978117 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:42:24.978042    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f35cfe0-96eb-4b73-84d1-386930a8b26d" path="/var/lib/kubelet/pods/6f35cfe0-96eb-4b73-84d1-386930a8b26d/volumes"
Sep 10 23:43:03.102136 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.102094    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"]
Sep 10 23:43:03.102745 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.102722    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f35cfe0-96eb-4b73-84d1-386930a8b26d" containerName="authorino"
Sep 10 23:43:03.102787 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.102752    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f35cfe0-96eb-4b73-84d1-386930a8b26d" containerName="authorino"
Sep 10 23:43:03.102857 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.102845    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f35cfe0-96eb-4b73-84d1-386930a8b26d" containerName="authorino"
Sep 10 23:43:03.106217 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.106197    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:03.108247 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.108219    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 23:43:03.108384 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.108348    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bgpsq\""
Sep 10 23:43:03.116628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.116603    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"]
Sep 10 23:43:03.184237 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.184201    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plg6m\" (UniqueName: \"kubernetes.io/projected/a66761c6-89c6-41fd-a2f2-5846a082df0d-kube-api-access-plg6m\") pod \"payload-pre-processing-59fbc47bf8-67mjh\" (UID: \"a66761c6-89c6-41fd-a2f2-5846a082df0d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:03.184430 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.184251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66761c6-89c6-41fd-a2f2-5846a082df0d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-67mjh\" (UID: \"a66761c6-89c6-41fd-a2f2-5846a082df0d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:03.285342 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.285303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plg6m\" (UniqueName: \"kubernetes.io/projected/a66761c6-89c6-41fd-a2f2-5846a082df0d-kube-api-access-plg6m\") pod \"payload-pre-processing-59fbc47bf8-67mjh\" (UID: \"a66761c6-89c6-41fd-a2f2-5846a082df0d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:03.285342 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.285342    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66761c6-89c6-41fd-a2f2-5846a082df0d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-67mjh\" (UID: \"a66761c6-89c6-41fd-a2f2-5846a082df0d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:03.286066 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.286009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66761c6-89c6-41fd-a2f2-5846a082df0d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-67mjh\" (UID: \"a66761c6-89c6-41fd-a2f2-5846a082df0d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:03.293047 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.293017    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plg6m\" (UniqueName: \"kubernetes.io/projected/a66761c6-89c6-41fd-a2f2-5846a082df0d-kube-api-access-plg6m\") pod \"payload-pre-processing-59fbc47bf8-67mjh\" (UID: \"a66761c6-89c6-41fd-a2f2-5846a082df0d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:03.419232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.419186    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:03.691378 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.691293    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"]
Sep 10 23:43:03.694345 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:43:03.694319    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66761c6_89c6_41fd_a2f2_5846a082df0d.slice/crio-e0e1500c12bfb6cc91f1d7d6c37fc95fc10af86ecc92d0be141e42aa3da16c9f WatchSource:0}: Error finding container e0e1500c12bfb6cc91f1d7d6c37fc95fc10af86ecc92d0be141e42aa3da16c9f: Status 404 returned error can't find the container with id e0e1500c12bfb6cc91f1d7d6c37fc95fc10af86ecc92d0be141e42aa3da16c9f
Sep 10 23:43:03.827884 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:03.827843    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh" event={"ID":"a66761c6-89c6-41fd-a2f2-5846a082df0d","Type":"ContainerStarted","Data":"e0e1500c12bfb6cc91f1d7d6c37fc95fc10af86ecc92d0be141e42aa3da16c9f"}
Sep 10 23:43:06.862967 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:06.862916    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh" event={"ID":"a66761c6-89c6-41fd-a2f2-5846a082df0d","Type":"ContainerStarted","Data":"38cf3f708c6649118a0350e447ce3b9f9c00cde5da328f70d249bb79b0690427"}
Sep 10 23:43:06.863421 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:06.863160    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:06.880852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:06.880801    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh" podStartSLOduration=0.970221714 podStartE2EDuration="3.880785623s" totalImagesPullingTime="2.910563909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:43:06.880086875 +0000 UTC m=+600.533807064" watchObservedRunningTime="2026-09-10 23:43:06.880785623 +0000 UTC m=+600.534505795"
Sep 10 23:43:06.971378 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:06.971352    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:43:06.982312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:06.982280    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:43:06.990326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:06.990299    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:43:06.992759 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:06.992675    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:43:12.892410 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:12.892374    2569 generic.go:356] "Generic (PLEG): container finished" podID="1ddd5db5-9ef6-4dea-a625-36023eb3e937" containerID="3c43aa72679e666681aa4bf4e95e645ac4a19515563dd4d6816735e6bcf1f9ff" exitCode=0
Sep 10 23:43:12.892832 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:12.892434    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" event={"ID":"1ddd5db5-9ef6-4dea-a625-36023eb3e937","Type":"ContainerDied","Data":"3c43aa72679e666681aa4bf4e95e645ac4a19515563dd4d6816735e6bcf1f9ff"}
Sep 10 23:43:12.892832 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:12.892788    2569 scope.go:117] "RemoveContainer" containerID="3c43aa72679e666681aa4bf4e95e645ac4a19515563dd4d6816735e6bcf1f9ff"
Sep 10 23:43:12.893763 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:12.893738    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:43:12.893816 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:12.893802    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:43:12.893860 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:12.893833    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:43:13.898928 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:13.898894    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" event={"ID":"1ddd5db5-9ef6-4dea-a625-36023eb3e937","Type":"ContainerStarted","Data":"482d140ec5be935652b61846357671be73a481ad69b0cb1ce3c1a568469c03c8"}
Sep 10 23:43:13.951038 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:13.950991    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:43:13.951379 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:13.951348    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n8b68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 10 23:43:13.951490 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:13.951416    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" podUID="1ddd5db5-9ef6-4dea-a625-36023eb3e937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 10 23:43:14.950949 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:14.950893    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n8b68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 10 23:43:14.951553 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:14.951521    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" podUID="1ddd5db5-9ef6-4dea-a625-36023eb3e937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 10 23:43:15.951055 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:15.951017    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n8b68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 10 23:43:15.951756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:15.951723    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" podUID="1ddd5db5-9ef6-4dea-a625-36023eb3e937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 10 23:43:16.951799 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:16.951760    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n8b68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 10 23:43:16.952442 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:16.952407    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" podUID="1ddd5db5-9ef6-4dea-a625-36023eb3e937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 10 23:43:17.870918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:17.870887    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-67mjh"
Sep 10 23:43:17.953961 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:17.951089    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-n8b68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 10 23:43:17.953961 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:17.951181    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68" podUID="1ddd5db5-9ef6-4dea-a625-36023eb3e937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 10 23:43:18.954650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:18.954622    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:43:18.955106 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:18.954867    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:43:18.955587 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:18.955571    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-n8b68"
Sep 10 23:43:24.399254 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.399215    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"]
Sep 10 23:43:24.407512 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.407486    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.410202 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.410174    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 23:43:24.410406 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.410240    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2vpzq\""
Sep 10 23:43:24.410406 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.410322    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 23:43:24.410406 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.410361    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 23:43:24.412852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.412827    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"]
Sep 10 23:43:24.470404 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.470371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.470582 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.470427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.470582 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.470483    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.470582 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.470512    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.470582 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.470547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71cf3014-6733-413a-9ddc-2e988c7338e7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.470582 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.470575    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wms2v\" (UniqueName: \"kubernetes.io/projected/71cf3014-6733-413a-9ddc-2e988c7338e7-kube-api-access-wms2v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.571585 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.571546    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.571770 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.571597    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.571770 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.571730    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.571866 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.571783    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.571866 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.571852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71cf3014-6733-413a-9ddc-2e988c7338e7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.572004 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.571892    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wms2v\" (UniqueName: \"kubernetes.io/projected/71cf3014-6733-413a-9ddc-2e988c7338e7-kube-api-access-wms2v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.572084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.572061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.572217 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.572197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.572455 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.572432    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.573981 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.573961    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71cf3014-6733-413a-9ddc-2e988c7338e7-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.574305 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.574290    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71cf3014-6733-413a-9ddc-2e988c7338e7-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.579058 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.579036    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wms2v\" (UniqueName: \"kubernetes.io/projected/71cf3014-6733-413a-9ddc-2e988c7338e7-kube-api-access-wms2v\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv\" (UID: \"71cf3014-6733-413a-9ddc-2e988c7338e7\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.718830 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.718735    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:24.991620 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:24.991593    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"]
Sep 10 23:43:24.993998 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:43:24.993972    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71cf3014_6733_413a_9ddc_2e988c7338e7.slice/crio-408cb83b404c7b97f4d7b3bfbbe6ee0a3e7d7cc4ac4415fb358a1442a6603d6e WatchSource:0}: Error finding container 408cb83b404c7b97f4d7b3bfbbe6ee0a3e7d7cc4ac4415fb358a1442a6603d6e: Status 404 returned error can't find the container with id 408cb83b404c7b97f4d7b3bfbbe6ee0a3e7d7cc4ac4415fb358a1442a6603d6e
Sep 10 23:43:25.957968 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:25.957910    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv" event={"ID":"71cf3014-6733-413a-9ddc-2e988c7338e7","Type":"ContainerStarted","Data":"408cb83b404c7b97f4d7b3bfbbe6ee0a3e7d7cc4ac4415fb358a1442a6603d6e"}
Sep 10 23:43:26.964886 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:26.964855    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv" event={"ID":"71cf3014-6733-413a-9ddc-2e988c7338e7","Type":"ContainerStarted","Data":"6f88eca449698a328759dcf9faa844b30a70b0654205d379ca9318f7cfdd197d"}
Sep 10 23:43:26.979729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:26.979675    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv" podStartSLOduration=1.868533345 podStartE2EDuration="2.979657943s" totalImagesPullingTime="1.111124598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:43:26.979081461 +0000 UTC m=+620.632801635" watchObservedRunningTime="2026-09-10 23:43:26.979657943 +0000 UTC m=+620.633378116"
Sep 10 23:43:34.718971 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:34.718861    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:34.719404 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:34.718976    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:34.731654 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:34.731632    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:43:35.009370 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:43:35.009290    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv"
Sep 10 23:45:00.143836 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:00.143754    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn"]
Sep 10 23:45:00.147224 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:00.147208    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn"
Sep 10 23:45:00.149157 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:00.149135    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:45:00.152379 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:00.152352    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn"]
Sep 10 23:45:00.159045 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:00.159024    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn"
Sep 10 23:45:00.429239 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:00.429160    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn"]
Sep 10 23:45:00.433996 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:45:00.433921    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8e97a4_b36c_4fa5_b439_85c7b0876202.slice/crio-923d6b169c8e376d1ca3bf991adc27d521757403f804130d6357b7bf2fd46ee7 WatchSource:0}: Error finding container 923d6b169c8e376d1ca3bf991adc27d521757403f804130d6357b7bf2fd46ee7: Status 404 returned error can't find the container with id 923d6b169c8e376d1ca3bf991adc27d521757403f804130d6357b7bf2fd46ee7
Sep 10 23:45:00.435827 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:00.435808    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 23:45:01.417650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:01.417612    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn" event={"ID":"1c8e97a4-b36c-4fa5-b439-85c7b0876202","Type":"ContainerStarted","Data":"923d6b169c8e376d1ca3bf991adc27d521757403f804130d6357b7bf2fd46ee7"}
Sep 10 23:45:03.430910 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:03.430875    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn" event={"ID":"1c8e97a4-b36c-4fa5-b439-85c7b0876202","Type":"ContainerStarted","Data":"75ba8c7bc0bd1798ceaf4d899625fd4b3e4c8d84e1c265e572c587c7442aa76d"}
Sep 10 23:45:03.444827 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:03.444770    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn" podStartSLOduration=1.1316331370000001 podStartE2EDuration="3.444751429s" totalImagesPullingTime="2.313118292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:45:03.444160984 +0000 UTC m=+717.097881157" watchObservedRunningTime="2026-09-10 23:45:03.444751429 +0000 UTC m=+717.098471599"
Sep 10 23:45:04.437951 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:04.437906    2569 generic.go:356] "Generic (PLEG): container finished" podID="1c8e97a4-b36c-4fa5-b439-85c7b0876202" containerID="75ba8c7bc0bd1798ceaf4d899625fd4b3e4c8d84e1c265e572c587c7442aa76d" exitCode=0
Sep 10 23:45:04.438359 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:04.437994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn" event={"ID":"1c8e97a4-b36c-4fa5-b439-85c7b0876202","Type":"ContainerDied","Data":"75ba8c7bc0bd1798ceaf4d899625fd4b3e4c8d84e1c265e572c587c7442aa76d"}
Sep 10 23:45:05.574215 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:05.574190    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn"
Sep 10 23:45:06.450160 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:06.450128    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn"
Sep 10 23:45:06.450327 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:06.450158    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818065-m6hmn" event={"ID":"1c8e97a4-b36c-4fa5-b439-85c7b0876202","Type":"ContainerDied","Data":"923d6b169c8e376d1ca3bf991adc27d521757403f804130d6357b7bf2fd46ee7"}
Sep 10 23:45:06.450327 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:06.450203    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="923d6b169c8e376d1ca3bf991adc27d521757403f804130d6357b7bf2fd46ee7"
Sep 10 23:45:38.155695 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.155660    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-668cdb6579-l44hr"]
Sep 10 23:45:38.156204 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.156048    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c8e97a4-b36c-4fa5-b439-85c7b0876202" containerName="cleanup"
Sep 10 23:45:38.156204 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.156061    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c8e97a4-b36c-4fa5-b439-85c7b0876202" containerName="cleanup"
Sep 10 23:45:38.156204 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.156128    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c8e97a4-b36c-4fa5-b439-85c7b0876202" containerName="cleanup"
Sep 10 23:45:38.164512 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.164485    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-668cdb6579-l44hr"
Sep 10 23:45:38.165586 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.165561    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-668cdb6579-l44hr"]
Sep 10 23:45:38.337649 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.337608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8dbce6c-773e-41b1-9c53-cd025ac3246c-tls-cert\") pod \"authorino-668cdb6579-l44hr\" (UID: \"e8dbce6c-773e-41b1-9c53-cd025ac3246c\") " pod="kuadrant-system/authorino-668cdb6579-l44hr"
Sep 10 23:45:38.337833 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.337659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9sxv\" (UniqueName: \"kubernetes.io/projected/e8dbce6c-773e-41b1-9c53-cd025ac3246c-kube-api-access-g9sxv\") pod \"authorino-668cdb6579-l44hr\" (UID: \"e8dbce6c-773e-41b1-9c53-cd025ac3246c\") " pod="kuadrant-system/authorino-668cdb6579-l44hr"
Sep 10 23:45:38.439251 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.439151    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8dbce6c-773e-41b1-9c53-cd025ac3246c-tls-cert\") pod \"authorino-668cdb6579-l44hr\" (UID: \"e8dbce6c-773e-41b1-9c53-cd025ac3246c\") " pod="kuadrant-system/authorino-668cdb6579-l44hr"
Sep 10 23:45:38.439251 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.439219    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9sxv\" (UniqueName: \"kubernetes.io/projected/e8dbce6c-773e-41b1-9c53-cd025ac3246c-kube-api-access-g9sxv\") pod \"authorino-668cdb6579-l44hr\" (UID: \"e8dbce6c-773e-41b1-9c53-cd025ac3246c\") " pod="kuadrant-system/authorino-668cdb6579-l44hr"
Sep 10 23:45:38.441603 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.441573    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e8dbce6c-773e-41b1-9c53-cd025ac3246c-tls-cert\") pod \"authorino-668cdb6579-l44hr\" (UID: \"e8dbce6c-773e-41b1-9c53-cd025ac3246c\") " pod="kuadrant-system/authorino-668cdb6579-l44hr"
Sep 10 23:45:38.446264 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.446239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9sxv\" (UniqueName: \"kubernetes.io/projected/e8dbce6c-773e-41b1-9c53-cd025ac3246c-kube-api-access-g9sxv\") pod \"authorino-668cdb6579-l44hr\" (UID: \"e8dbce6c-773e-41b1-9c53-cd025ac3246c\") " pod="kuadrant-system/authorino-668cdb6579-l44hr"
Sep 10 23:45:38.474233 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.474187    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-668cdb6579-l44hr"
Sep 10 23:45:38.742735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:38.742710    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-668cdb6579-l44hr"]
Sep 10 23:45:38.745474 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:45:38.745445    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8dbce6c_773e_41b1_9c53_cd025ac3246c.slice/crio-b406880765efe3faa74651409a0aa779a6e26da529b65e989d7bb0f720e6ead4 WatchSource:0}: Error finding container b406880765efe3faa74651409a0aa779a6e26da529b65e989d7bb0f720e6ead4: Status 404 returned error can't find the container with id b406880765efe3faa74651409a0aa779a6e26da529b65e989d7bb0f720e6ead4
Sep 10 23:45:39.608408 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:39.608320    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-668cdb6579-l44hr" event={"ID":"e8dbce6c-773e-41b1-9c53-cd025ac3246c","Type":"ContainerStarted","Data":"8319746293dd0e45238091caaf253131e8dabd8439590dc25e130df228b849b1"}
Sep 10 23:45:39.608408 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:39.608364    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-668cdb6579-l44hr" event={"ID":"e8dbce6c-773e-41b1-9c53-cd025ac3246c","Type":"ContainerStarted","Data":"b406880765efe3faa74651409a0aa779a6e26da529b65e989d7bb0f720e6ead4"}
Sep 10 23:45:39.627545 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:39.627449    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-668cdb6579-l44hr" podStartSLOduration=1.226094256 podStartE2EDuration="1.627427692s" totalImagesPullingTime="401.333436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:45:39.626801718 +0000 UTC m=+753.280521892" watchObservedRunningTime="2026-09-10 23:45:39.627427692 +0000 UTC m=+753.281147866"
Sep 10 23:45:39.655256 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:39.655216    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-594d848786-h72mj"]
Sep 10 23:45:39.655469 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:39.655435    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-594d848786-h72mj" podUID="9968035a-b5fe-498b-8888-150bd2305c7a" containerName="authorino" containerID="cri-o://b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb" gracePeriod=30
Sep 10 23:45:39.923146 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:39.923119    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:45:40.058280 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.058241    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slkgq\" (UniqueName: \"kubernetes.io/projected/9968035a-b5fe-498b-8888-150bd2305c7a-kube-api-access-slkgq\") pod \"9968035a-b5fe-498b-8888-150bd2305c7a\" (UID: \"9968035a-b5fe-498b-8888-150bd2305c7a\") "
Sep 10 23:45:40.058440 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.058303    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9968035a-b5fe-498b-8888-150bd2305c7a-tls-cert\") pod \"9968035a-b5fe-498b-8888-150bd2305c7a\" (UID: \"9968035a-b5fe-498b-8888-150bd2305c7a\") "
Sep 10 23:45:40.060375 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.060344    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9968035a-b5fe-498b-8888-150bd2305c7a-kube-api-access-slkgq" (OuterVolumeSpecName: "kube-api-access-slkgq") pod "9968035a-b5fe-498b-8888-150bd2305c7a" (UID: "9968035a-b5fe-498b-8888-150bd2305c7a"). InnerVolumeSpecName "kube-api-access-slkgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:45:40.070458 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.070429    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9968035a-b5fe-498b-8888-150bd2305c7a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9968035a-b5fe-498b-8888-150bd2305c7a" (UID: "9968035a-b5fe-498b-8888-150bd2305c7a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 23:45:40.159504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.159401    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slkgq\" (UniqueName: \"kubernetes.io/projected/9968035a-b5fe-498b-8888-150bd2305c7a-kube-api-access-slkgq\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:45:40.159504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.159450    2569 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9968035a-b5fe-498b-8888-150bd2305c7a-tls-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:45:40.615263 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.615228    2569 generic.go:356] "Generic (PLEG): container finished" podID="9968035a-b5fe-498b-8888-150bd2305c7a" containerID="b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb" exitCode=0
Sep 10 23:45:40.615664 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.615272    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594d848786-h72mj"
Sep 10 23:45:40.615664 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.615314    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594d848786-h72mj" event={"ID":"9968035a-b5fe-498b-8888-150bd2305c7a","Type":"ContainerDied","Data":"b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb"}
Sep 10 23:45:40.615664 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.615356    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-594d848786-h72mj" event={"ID":"9968035a-b5fe-498b-8888-150bd2305c7a","Type":"ContainerDied","Data":"fdda4a694e681d29d1daec7ab912973a60dbbb7003e171384deb17588326d492"}
Sep 10 23:45:40.615664 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.615368    2569 scope.go:117] "RemoveContainer" containerID="b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb"
Sep 10 23:45:40.625825 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.625803    2569 scope.go:117] "RemoveContainer" containerID="b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb"
Sep 10 23:45:40.626158 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:45:40.626140    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb\": container with ID starting with b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb not found: ID does not exist" containerID="b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb"
Sep 10 23:45:40.626232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.626166    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb"} err="failed to get container status \"b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb\": rpc error: code = NotFound desc = could not find container \"b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb\": container with ID starting with b77346abfc03a7741041ef7405575f9125c94064495f0f28345807d806be21fb not found: ID does not exist"
Sep 10 23:45:40.636054 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.636031    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-594d848786-h72mj"]
Sep 10 23:45:40.642019 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.641998    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-594d848786-h72mj"]
Sep 10 23:45:40.978662 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:40.978626    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9968035a-b5fe-498b-8888-150bd2305c7a" path="/var/lib/kubelet/pods/9968035a-b5fe-498b-8888-150bd2305c7a/volumes"
Sep 10 23:45:56.983257 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:56.983225    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"]
Sep 10 23:45:56.983724 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:56.983644    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9968035a-b5fe-498b-8888-150bd2305c7a" containerName="authorino"
Sep 10 23:45:56.983724 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:56.983658    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9968035a-b5fe-498b-8888-150bd2305c7a" containerName="authorino"
Sep 10 23:45:56.983804 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:56.983732    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9968035a-b5fe-498b-8888-150bd2305c7a" containerName="authorino"
Sep 10 23:45:56.988229 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:56.988209    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:56.990244 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:56.990221    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-487960-openshift-default-dockercfg-qdpgb\""
Sep 10 23:45:57.000309 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.000283    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"]
Sep 10 23:45:57.112005 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.111969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-credential-socket\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.112005 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.112009    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-certs\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.112225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.112033    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-podinfo\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.112225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.112059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istiod-ca-cert\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.112225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.112163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-data\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.112225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.112190    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-token\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.112225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.112215    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-envoy\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.112467 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.112254    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-socket\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.112467 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.112361    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxk85\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-kube-api-access-dxk85\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213345 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-socket\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213531 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213366    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxk85\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-kube-api-access-dxk85\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213531 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-credential-socket\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213531 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-certs\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213531 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213467    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-podinfo\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213753 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istiod-ca-cert\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213753 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213727    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-data\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213863 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213758    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-token\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213863 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213760    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-socket\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213863 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-envoy\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213863 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213788    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-certs\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.213863 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.213852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-credential-socket\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.214142 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.214114    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-data\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.214298 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.214279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istiod-ca-cert\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.215864 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.215841    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-envoy\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.215964 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.215912    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-podinfo\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.220686 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.220659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxk85\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-kube-api-access-dxk85\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.221158 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.221140    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-token\") pod \"e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") " pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.301347 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.301265    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:57.578197 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.578173    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"]
Sep 10 23:45:57.580331 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:45:57.580302    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5591bae4_a799_462a_b00d_fe0dff4cdbb3.slice/crio-56ccdf3aae433e5d7f6ee8e1bbbff853b9ab8860e19f24d457d0de0f1d94173f WatchSource:0}: Error finding container 56ccdf3aae433e5d7f6ee8e1bbbff853b9ab8860e19f24d457d0de0f1d94173f: Status 404 returned error can't find the container with id 56ccdf3aae433e5d7f6ee8e1bbbff853b9ab8860e19f24d457d0de0f1d94173f
Sep 10 23:45:57.582479 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.582447    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:45:57.582577 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.582526    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:45:57.582577 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.582566    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:45:57.699012 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.698971    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" event={"ID":"5591bae4-a799-462a-b00d-fe0dff4cdbb3","Type":"ContainerStarted","Data":"627a2ee07a45d59644d968c83fef56fd09e4c29727a6c16168e403b21b0b7ab7"}
Sep 10 23:45:57.699205 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.699031    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" event={"ID":"5591bae4-a799-462a-b00d-fe0dff4cdbb3","Type":"ContainerStarted","Data":"56ccdf3aae433e5d7f6ee8e1bbbff853b9ab8860e19f24d457d0de0f1d94173f"}
Sep 10 23:45:57.717390 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:57.717332    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podStartSLOduration=1.717309628 podStartE2EDuration="1.717309628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:45:57.715303085 +0000 UTC m=+771.369023259" watchObservedRunningTime="2026-09-10 23:45:57.717309628 +0000 UTC m=+771.371029802"
Sep 10 23:45:58.302335 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.302292    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:58.307550 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.307525    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:58.563080 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.562999    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7821-b47e11"]
Sep 10 23:45:58.566671 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.566655    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:45:58.568516 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.568493    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9kl27\""
Sep 10 23:45:58.572196 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.572169    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7821-b47e11"]
Sep 10 23:45:58.627187 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.627158    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdtpb\" (UniqueName: \"kubernetes.io/projected/dd14e34b-20ec-4a3e-a9b9-ed8278851bba-kube-api-access-gdtpb\") pod \"test-curl-7821-b47e11\" (UID: \"dd14e34b-20ec-4a3e-a9b9-ed8278851bba\") " pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:45:58.706899 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.706870    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:58.707997 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.707978    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:45:58.728054 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.728022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdtpb\" (UniqueName: \"kubernetes.io/projected/dd14e34b-20ec-4a3e-a9b9-ed8278851bba-kube-api-access-gdtpb\") pod \"test-curl-7821-b47e11\" (UID: \"dd14e34b-20ec-4a3e-a9b9-ed8278851bba\") " pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:45:58.739222 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.739197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdtpb\" (UniqueName: \"kubernetes.io/projected/dd14e34b-20ec-4a3e-a9b9-ed8278851bba-kube-api-access-gdtpb\") pod \"test-curl-7821-b47e11\" (UID: \"dd14e34b-20ec-4a3e-a9b9-ed8278851bba\") " pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:45:58.877247 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:58.877224    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:45:59.146695 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:59.146670    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7821-b47e11"]
Sep 10 23:45:59.148849 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:45:59.148818    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd14e34b_20ec_4a3e_a9b9_ed8278851bba.slice/crio-095da07dec21d6b567a99168bd0330c4661443e27b2c3bbfa61b1ad72c05681c WatchSource:0}: Error finding container 095da07dec21d6b567a99168bd0330c4661443e27b2c3bbfa61b1ad72c05681c: Status 404 returned error can't find the container with id 095da07dec21d6b567a99168bd0330c4661443e27b2c3bbfa61b1ad72c05681c
Sep 10 23:45:59.712990 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:45:59.712899    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7821-b47e11" event={"ID":"dd14e34b-20ec-4a3e-a9b9-ed8278851bba","Type":"ContainerStarted","Data":"095da07dec21d6b567a99168bd0330c4661443e27b2c3bbfa61b1ad72c05681c"}
Sep 10 23:46:01.461094 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.461065    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"]
Sep 10 23:46:01.466371 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.466347    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:01.469029 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.469005    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-487960\""
Sep 10 23:46:01.469147 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.469005    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-487960-dockercfg-wfgv2\""
Sep 10 23:46:01.475699 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.475672    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"]
Sep 10 23:46:01.555390 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.555356    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj\" (UID: \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\") " pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:01.555390 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.555398    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8s6v\" (UniqueName: \"kubernetes.io/projected/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-kube-api-access-k8s6v\") pod \"payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj\" (UID: \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\") " pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:01.656790 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.656744    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj\" (UID: \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\") " pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:01.656790 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.656795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8s6v\" (UniqueName: \"kubernetes.io/projected/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-kube-api-access-k8s6v\") pod \"payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj\" (UID: \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\") " pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:01.657775 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.657747    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj\" (UID: \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\") " pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:01.680150 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.680073    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8s6v\" (UniqueName: \"kubernetes.io/projected/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-kube-api-access-k8s6v\") pod \"payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj\" (UID: \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\") " pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:01.727221 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.727179    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7821-b47e11" event={"ID":"dd14e34b-20ec-4a3e-a9b9-ed8278851bba","Type":"ContainerStarted","Data":"bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a"}
Sep 10 23:46:01.742822 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.742757    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7821-b47e11" podStartSLOduration=1.474032819 podStartE2EDuration="3.742734195s" totalImagesPullingTime="2.268701376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:45:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:01.739570053 +0000 UTC m=+775.393290228" watchObservedRunningTime="2026-09-10 23:46:01.742734195 +0000 UTC m=+775.396454369"
Sep 10 23:46:01.793343 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:01.793307    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:02.077157 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:02.077124    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"]
Sep 10 23:46:02.077876 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:46:02.077851    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ed38e3_dfb1_4827_872e_5ecaa74e9c29.slice/crio-5043cf9f71f9509ee91bfbba60b72af964ec62810a2ae4e479264c642520836a WatchSource:0}: Error finding container 5043cf9f71f9509ee91bfbba60b72af964ec62810a2ae4e479264c642520836a: Status 404 returned error can't find the container with id 5043cf9f71f9509ee91bfbba60b72af964ec62810a2ae4e479264c642520836a
Sep 10 23:46:02.308979 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:02.308919    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7821-b47e11"]
Sep 10 23:46:02.314689 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:02.314658    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7821-b47e11"]
Sep 10 23:46:02.734175 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:02.734142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj" event={"ID":"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29","Type":"ContainerStarted","Data":"0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b"}
Sep 10 23:46:02.734662 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:02.734190    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj" event={"ID":"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29","Type":"ContainerStarted","Data":"5043cf9f71f9509ee91bfbba60b72af964ec62810a2ae4e479264c642520836a"}
Sep 10 23:46:02.734662 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:02.734272    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:02.734662 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:46:02.734487    2569 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7821-b47e11" container="test-curl-7821-b47e11"
Sep 10 23:46:02.754140 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:02.754089    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj" podStartSLOduration=1.754070365 podStartE2EDuration="1.754070365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:02.751836334 +0000 UTC m=+776.405556518" watchObservedRunningTime="2026-09-10 23:46:02.754070365 +0000 UTC m=+776.407790539"
Sep 10 23:46:03.740167 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:03.740112    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7821-b47e11" podUID="dd14e34b-20ec-4a3e-a9b9-ed8278851bba" containerName="test-curl-7821-b47e11" containerID="cri-o://bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a" gracePeriod=2
Sep 10 23:46:04.777521 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.777484    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"]
Sep 10 23:46:04.778150 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.778113    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd14e34b-20ec-4a3e-a9b9-ed8278851bba" containerName="test-curl-7821-b47e11"
Sep 10 23:46:04.778150 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.778151    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd14e34b-20ec-4a3e-a9b9-ed8278851bba" containerName="test-curl-7821-b47e11"
Sep 10 23:46:04.778345 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.778260    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd14e34b-20ec-4a3e-a9b9-ed8278851bba" containerName="test-curl-7821-b47e11"
Sep 10 23:46:04.781490 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.781471    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.783571 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.783547    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9dfc47f0-openshift-default-dockercfg-wbnwf\""
Sep 10 23:46:04.793764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.793741    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"]
Sep 10 23:46:04.888242 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-podinfo\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.888443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888285    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-socket\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.888443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888333    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-credential-socket\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.888443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888355    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-envoy\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.888443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888433    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-certs\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.888646 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-data\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.888646 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888479    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-token\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.888646 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888519    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istiod-ca-cert\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.888646 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.888598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clszl\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-kube-api-access-clszl\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.989521 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-certs\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.989521 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989522    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-data\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.989759 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989643    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-token\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.989759 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istiod-ca-cert\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.989759 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989752    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clszl\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-kube-api-access-clszl\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.989919 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989789    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-podinfo\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.989919 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-socket\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.989919 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-certs\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.990152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.989919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-credential-socket\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.990152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.990017    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-data\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.990152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.990047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-envoy\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.990339 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.990319    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-credential-socket\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.990392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.990368    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-socket\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.990526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.990506    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istiod-ca-cert\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.992127 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.992105    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-envoy\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.992598 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.992581    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-podinfo\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.997048 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.997025    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-token\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:04.997267 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:04.997243    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clszl\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-kube-api-access-clszl\") pod \"e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") " pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:05.093356 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.093266    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:05.370284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.370261    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"]
Sep 10 23:46:05.372960 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:46:05.372912    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce99e366_8e62_4d15_bebf_bb9e33d501ba.slice/crio-99b16afc88033efddbc4997cf5fbcd00604b296015c99a308fcd63f36a981b52 WatchSource:0}: Error finding container 99b16afc88033efddbc4997cf5fbcd00604b296015c99a308fcd63f36a981b52: Status 404 returned error can't find the container with id 99b16afc88033efddbc4997cf5fbcd00604b296015c99a308fcd63f36a981b52
Sep 10 23:46:05.375328 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.375292    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:05.375431 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.375375    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:05.375431 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.375423    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:05.410584 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.410556    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"]
Sep 10 23:46:05.415608 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.415577    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.418081 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.417835    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-abe3de-openshift-default-dockercfg-h7b2h\""
Sep 10 23:46:05.427349 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.427321    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"]
Sep 10 23:46:05.495426 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-certs\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.495426 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-podinfo\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.495641 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-data\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.495641 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-token\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.495641 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495567    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-socket\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.495641 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-envoy\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.495641 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495628    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-credential-socket\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.495819 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495735    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d27ee9-2167-4cc3-84af-245a86e7672c-istiod-ca-cert\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.495819 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.495802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcfcv\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-kube-api-access-bcfcv\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.596856 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.596807    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-token\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.596869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-socket\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.596896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-envoy\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.596923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-credential-socket\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.596990    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d27ee9-2167-4cc3-84af-245a86e7672c-istiod-ca-cert\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.597043    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcfcv\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-kube-api-access-bcfcv\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597376 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.597127    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-certs\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597376 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.597152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-podinfo\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597376 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.597184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-data\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.597438    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-credential-socket\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.597466    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-socket\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.597499    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-data\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.597680 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.597629    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-certs\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.598090 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.598068    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d27ee9-2167-4cc3-84af-245a86e7672c-istiod-ca-cert\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.599744 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.599717    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-podinfo\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.599852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.599832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-envoy\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.604033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.604007    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-token\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.604896 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.604876    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcfcv\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-kube-api-access-bcfcv\") pod \"e2e-autores-abe3de-openshift-default-58cf495b46-xstf8\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") " pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.742060 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.741974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:05.751954 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.751906    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" event={"ID":"ce99e366-8e62-4d15-bebf-bb9e33d501ba","Type":"ContainerStarted","Data":"7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705"}
Sep 10 23:46:05.752068 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.751981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" event={"ID":"ce99e366-8e62-4d15-bebf-bb9e33d501ba","Type":"ContainerStarted","Data":"99b16afc88033efddbc4997cf5fbcd00604b296015c99a308fcd63f36a981b52"}
Sep 10 23:46:05.771015 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.770957    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" podStartSLOduration=1.770915985 podStartE2EDuration="1.770915985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:05.769074858 +0000 UTC m=+779.422795031" watchObservedRunningTime="2026-09-10 23:46:05.770915985 +0000 UTC m=+779.424636160"
Sep 10 23:46:05.920639 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:05.920614    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:46:06.001428 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.001354    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdtpb\" (UniqueName: \"kubernetes.io/projected/dd14e34b-20ec-4a3e-a9b9-ed8278851bba-kube-api-access-gdtpb\") pod \"dd14e34b-20ec-4a3e-a9b9-ed8278851bba\" (UID: \"dd14e34b-20ec-4a3e-a9b9-ed8278851bba\") "
Sep 10 23:46:06.003901 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.003863    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd14e34b-20ec-4a3e-a9b9-ed8278851bba-kube-api-access-gdtpb" (OuterVolumeSpecName: "kube-api-access-gdtpb") pod "dd14e34b-20ec-4a3e-a9b9-ed8278851bba" (UID: "dd14e34b-20ec-4a3e-a9b9-ed8278851bba"). InnerVolumeSpecName "kube-api-access-gdtpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:46:06.029547 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.029522    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"]
Sep 10 23:46:06.031648 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:46:06.031621    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d27ee9_2167_4cc3_84af_245a86e7672c.slice/crio-d2c15d43fb4043ac71126f8272fbd9b89435eeaf323616a0bd02377c538a544c WatchSource:0}: Error finding container d2c15d43fb4043ac71126f8272fbd9b89435eeaf323616a0bd02377c538a544c: Status 404 returned error can't find the container with id d2c15d43fb4043ac71126f8272fbd9b89435eeaf323616a0bd02377c538a544c
Sep 10 23:46:06.034632 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.034601    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:06.034738 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.034678    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:06.034738 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.034723    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:06.094008 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.093979    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:06.099623 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.099604    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:06.103182 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.103158    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdtpb\" (UniqueName: \"kubernetes.io/projected/dd14e34b-20ec-4a3e-a9b9-ed8278851bba-kube-api-access-gdtpb\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:06.758240 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.758205    2569 generic.go:356] "Generic (PLEG): container finished" podID="dd14e34b-20ec-4a3e-a9b9-ed8278851bba" containerID="bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a" exitCode=137
Sep 10 23:46:06.758420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.758247    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:46:06.758420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.758314    2569 scope.go:117] "RemoveContainer" containerID="bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a"
Sep 10 23:46:06.760193 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.760160    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" event={"ID":"13d27ee9-2167-4cc3-84af-245a86e7672c","Type":"ContainerStarted","Data":"4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1"}
Sep 10 23:46:06.760319 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.760206    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" event={"ID":"13d27ee9-2167-4cc3-84af-245a86e7672c","Type":"ContainerStarted","Data":"d2c15d43fb4043ac71126f8272fbd9b89435eeaf323616a0bd02377c538a544c"}
Sep 10 23:46:06.760467 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.760443    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:06.761663 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.761639    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:06.770819 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.770801    2569 scope.go:117] "RemoveContainer" containerID="bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a"
Sep 10 23:46:06.771511 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:46:06.771485    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a\": container with ID starting with bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a not found: ID does not exist" containerID="bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a"
Sep 10 23:46:06.771609 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.771520    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a"} err="failed to get container status \"bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a\": rpc error: code = NotFound desc = could not find container \"bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a\": container with ID starting with bc908101fdd3d4ea3700c6ecfc9dde8fbf0507ec43c73069a197f6fdf082ba4a not found: ID does not exist"
Sep 10 23:46:06.781771 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:46:06.781740    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7821-b47e11\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="dd14e34b-20ec-4a3e-a9b9-ed8278851bba" pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:46:06.782739 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.782700    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podStartSLOduration=1.782686422 podStartE2EDuration="1.782686422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:06.780077024 +0000 UTC m=+780.433797209" watchObservedRunningTime="2026-09-10 23:46:06.782686422 +0000 UTC m=+780.436406595"
Sep 10 23:46:06.985452 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:06.985422    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd14e34b-20ec-4a3e-a9b9-ed8278851bba" path="/var/lib/kubelet/pods/dd14e34b-20ec-4a3e-a9b9-ed8278851bba/volumes"
Sep 10 23:46:06.986261 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:46:06.986229    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7821-b47e11\" is forbidden: User \"system:node:ip-10-0-143-154.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-154.ec2.internal' and this object" podUID="dd14e34b-20ec-4a3e-a9b9-ed8278851bba" pod="opendatahub/test-curl-7821-b47e11"
Sep 10 23:46:07.743446 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:07.743408    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:07.758121 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:07.758084    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:07.769451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:07.769414    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:07.770691 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:07.770663    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:10.730301 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.730267    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"]
Sep 10 23:46:10.735728 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.735710    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:10.741957 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.738282    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-abe3de\""
Sep 10 23:46:10.741957 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.738517    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-abe3de-dockercfg-kvx44\""
Sep 10 23:46:10.754957 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.753056    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"]
Sep 10 23:46:10.757291 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.755901    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"]
Sep 10 23:46:10.784010 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.782837    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"]
Sep 10 23:46:10.784010 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.783150    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:10.886538 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.886497    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99d1ec48-7e67-4039-b33b-21b3372b5c2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn\" (UID: \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:10.886723 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.886567    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbbtn\" (UniqueName: \"kubernetes.io/projected/99d1ec48-7e67-4039-b33b-21b3372b5c2c-kube-api-access-wbbtn\") pod \"payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn\" (UID: \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:10.886723 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.886596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02198fef-fa7e-485a-954f-93d599cc85a3-plugins-config-volume\") pod \"payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd\" (UID: \"02198fef-fa7e-485a-954f-93d599cc85a3\") " pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:10.886723 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.886628    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnbxw\" (UniqueName: \"kubernetes.io/projected/02198fef-fa7e-485a-954f-93d599cc85a3-kube-api-access-cnbxw\") pod \"payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd\" (UID: \"02198fef-fa7e-485a-954f-93d599cc85a3\") " pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:10.987295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.987204    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99d1ec48-7e67-4039-b33b-21b3372b5c2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn\" (UID: \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:10.987295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.987245    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbbtn\" (UniqueName: \"kubernetes.io/projected/99d1ec48-7e67-4039-b33b-21b3372b5c2c-kube-api-access-wbbtn\") pod \"payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn\" (UID: \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:10.987295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.987262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02198fef-fa7e-485a-954f-93d599cc85a3-plugins-config-volume\") pod \"payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd\" (UID: \"02198fef-fa7e-485a-954f-93d599cc85a3\") " pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:10.987295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.987279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnbxw\" (UniqueName: \"kubernetes.io/projected/02198fef-fa7e-485a-954f-93d599cc85a3-kube-api-access-cnbxw\") pod \"payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd\" (UID: \"02198fef-fa7e-485a-954f-93d599cc85a3\") " pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:10.987998 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.987972    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99d1ec48-7e67-4039-b33b-21b3372b5c2c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn\" (UID: \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:10.988123 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:10.988009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02198fef-fa7e-485a-954f-93d599cc85a3-plugins-config-volume\") pod \"payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd\" (UID: \"02198fef-fa7e-485a-954f-93d599cc85a3\") " pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:11.003544 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.003512    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbbtn\" (UniqueName: \"kubernetes.io/projected/99d1ec48-7e67-4039-b33b-21b3372b5c2c-kube-api-access-wbbtn\") pod \"payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn\" (UID: \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:11.003803 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.003768    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnbxw\" (UniqueName: \"kubernetes.io/projected/02198fef-fa7e-485a-954f-93d599cc85a3-kube-api-access-cnbxw\") pod \"payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd\" (UID: \"02198fef-fa7e-485a-954f-93d599cc85a3\") " pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:11.071701 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.071664    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:11.095708 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.095676    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:11.344563 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.344443    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"]
Sep 10 23:46:11.410694 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.410666    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"]
Sep 10 23:46:11.412140 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:46:11.412115    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02198fef_fa7e_485a_954f_93d599cc85a3.slice/crio-0725a7b00c6cbb66746352e28df08fc1aae5b90a998261d2ffbcb4033e93b58a WatchSource:0}: Error finding container 0725a7b00c6cbb66746352e28df08fc1aae5b90a998261d2ffbcb4033e93b58a: Status 404 returned error can't find the container with id 0725a7b00c6cbb66746352e28df08fc1aae5b90a998261d2ffbcb4033e93b58a
Sep 10 23:46:11.796368 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.796323    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn" event={"ID":"99d1ec48-7e67-4039-b33b-21b3372b5c2c","Type":"ContainerStarted","Data":"4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976"}
Sep 10 23:46:11.796833 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.796383    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn" event={"ID":"99d1ec48-7e67-4039-b33b-21b3372b5c2c","Type":"ContainerStarted","Data":"7640dcd0d846cc1064d7060b28176aea59d5421d8c221a384fbcd94bd564babe"}
Sep 10 23:46:11.796833 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.796616    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:11.798291 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.798265    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd" event={"ID":"02198fef-fa7e-485a-954f-93d599cc85a3","Type":"ContainerStarted","Data":"c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec"}
Sep 10 23:46:11.798382 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.798313    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd" event={"ID":"02198fef-fa7e-485a-954f-93d599cc85a3","Type":"ContainerStarted","Data":"0725a7b00c6cbb66746352e28df08fc1aae5b90a998261d2ffbcb4033e93b58a"}
Sep 10 23:46:11.798382 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.798341    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:11.810817 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.810774    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn" podStartSLOduration=1.810759693 podStartE2EDuration="1.810759693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:11.81043963 +0000 UTC m=+785.464159804" watchObservedRunningTime="2026-09-10 23:46:11.810759693 +0000 UTC m=+785.464479865"
Sep 10 23:46:11.826318 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:11.826277    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd" podStartSLOduration=1.82626241 podStartE2EDuration="1.82626241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:11.825189806 +0000 UTC m=+785.478909982" watchObservedRunningTime="2026-09-10 23:46:11.82626241 +0000 UTC m=+785.479982582"
Sep 10 23:46:13.742813 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:13.742781    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:46:14.749151 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:14.749106    2569 patch_prober.go:28] interesting pod/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 23:46:14.749600 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:14.749183    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 23:46:16.828607 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:16.828509    2569 generic.go:356] "Generic (PLEG): container finished" podID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerID="627a2ee07a45d59644d968c83fef56fd09e4c29727a6c16168e403b21b0b7ab7" exitCode=0
Sep 10 23:46:16.828607 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:16.828586    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" event={"ID":"5591bae4-a799-462a-b00d-fe0dff4cdbb3","Type":"ContainerDied","Data":"627a2ee07a45d59644d968c83fef56fd09e4c29727a6c16168e403b21b0b7ab7"}
Sep 10 23:46:16.829058 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:16.828996    2569 scope.go:117] "RemoveContainer" containerID="627a2ee07a45d59644d968c83fef56fd09e4c29727a6c16168e403b21b0b7ab7"
Sep 10 23:46:16.830146 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:16.830122    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:16.830217 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:16.830190    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:16.830255 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:16.830219    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:17.837096 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:17.837057    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" event={"ID":"5591bae4-a799-462a-b00d-fe0dff4cdbb3","Type":"ContainerStarted","Data":"a812cb8ae666cd8c7ba86f006c90252038d2e23cb67faee05252c934a7607ab1"}
Sep 10 23:46:18.302767 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:18.302719    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:46:18.303007 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:18.302869    2569 patch_prober.go:28] interesting pod/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 23:46:18.303007 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:18.302914    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 23:46:19.302604 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:19.302555    2569 patch_prober.go:28] interesting pod/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 23:46:19.303124 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:19.302627    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 23:46:20.302626 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:20.302574    2569 patch_prober.go:28] interesting pod/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 23:46:20.303136 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:20.302650    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 23:46:21.301730 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:21.301683    2569 patch_prober.go:28] interesting pod/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 23:46:21.301975 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:21.301755    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 23:46:22.301849 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:22.301816    2569 patch_prober.go:28] interesting pod/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 23:46:22.302305 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:22.302279    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 23:46:22.806139 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:22.806099    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:46:22.807148 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:22.807104    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:46:23.302822 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:23.302626    2569 patch_prober.go:28] interesting pod/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 10 23:46:23.302822 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:23.302699    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 10 23:46:23.811554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:23.811142    2569 patch_prober.go:28] interesting pod/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 23:46:23.811554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:23.811221    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 23:46:24.307589 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:24.307408    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:46:24.308255 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:24.308178    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:46:24.309221 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:24.309142    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:46:25.201062 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.201028    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"]
Sep 10 23:46:25.208518 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.208485    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.211425 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.211400    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ea66ce-openshift-default-dockercfg-jz9xc\""
Sep 10 23:46:25.227309 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.227282    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"]
Sep 10 23:46:25.254886 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.254850    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-credential-socket\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.255061 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.254899    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-istio-token\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.255061 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.254996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d38d48-20f1-4730-887c-1457b6849524-istio-podinfo\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.255419 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.255090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-data\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.255419 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.255153    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-socket\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.255419 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.255185    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-certs\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.255419 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.255268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z9d7\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-kube-api-access-4z9d7\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.255419 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.255315    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d38d48-20f1-4730-887c-1457b6849524-istiod-ca-cert\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.255629 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.255420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-envoy\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356176 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356128    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-data\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356176 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356181    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-socket\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-certs\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z9d7\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-kube-api-access-4z9d7\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d38d48-20f1-4730-887c-1457b6849524-istiod-ca-cert\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356294    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-envoy\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356327    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-credential-socket\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356354    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-istio-token\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356389    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d38d48-20f1-4730-887c-1457b6849524-istio-podinfo\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356660    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-certs\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.356756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.356692    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-socket\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.357308 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.357258    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-data\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.357437 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.357390    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-credential-socket\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.357570 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.357549    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d38d48-20f1-4730-887c-1457b6849524-istiod-ca-cert\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.360211 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.360165    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d38d48-20f1-4730-887c-1457b6849524-istio-podinfo\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.361175 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.361145    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-envoy\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.364905 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.364856    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z9d7\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-kube-api-access-4z9d7\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.365013 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.364922    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-istio-token\") pod \"e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") " pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.520038 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.519943    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:25.821067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.821037    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"]
Sep 10 23:46:25.823682 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:46:25.823653    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d38d48_20f1_4730_887c_1457b6849524.slice/crio-843be2f5edc08b6ac376be2c4b1eb80ac6cec0205ad11470a1302473df73176b WatchSource:0}: Error finding container 843be2f5edc08b6ac376be2c4b1eb80ac6cec0205ad11470a1302473df73176b: Status 404 returned error can't find the container with id 843be2f5edc08b6ac376be2c4b1eb80ac6cec0205ad11470a1302473df73176b
Sep 10 23:46:25.826021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.825989    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:25.826123 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.826049    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:25.826123 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.826076    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:25.940138 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.939930    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" event={"ID":"91d38d48-20f1-4730-887c-1457b6849524","Type":"ContainerStarted","Data":"52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd"}
Sep 10 23:46:25.940325 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.940147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" event={"ID":"91d38d48-20f1-4730-887c-1457b6849524","Type":"ContainerStarted","Data":"843be2f5edc08b6ac376be2c4b1eb80ac6cec0205ad11470a1302473df73176b"}
Sep 10 23:46:25.959583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:25.959527    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" podStartSLOduration=0.959508114 podStartE2EDuration="959.508114ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:25.958320147 +0000 UTC m=+799.612040324" watchObservedRunningTime="2026-09-10 23:46:25.959508114 +0000 UTC m=+799.613228288"
Sep 10 23:46:26.520793 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.520709    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:26.525627 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.525603    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:26.945471 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.945439    2569 generic.go:356] "Generic (PLEG): container finished" podID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerID="7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705" exitCode=0
Sep 10 23:46:26.945640 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.945511    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" event={"ID":"ce99e366-8e62-4d15-bebf-bb9e33d501ba","Type":"ContainerDied","Data":"7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705"}
Sep 10 23:46:26.946049 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.946030    2569 scope.go:117] "RemoveContainer" containerID="7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705"
Sep 10 23:46:26.947207 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.947188    2569 generic.go:356] "Generic (PLEG): container finished" podID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerID="4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1" exitCode=0
Sep 10 23:46:26.947303 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.947264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" event={"ID":"13d27ee9-2167-4cc3-84af-245a86e7672c","Type":"ContainerDied","Data":"4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1"}
Sep 10 23:46:26.947303 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.947280    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:26.947400 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.947345    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:26.947400 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.947382    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:26.947471 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.947444    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:26.947787 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.947770    2569 scope.go:117] "RemoveContainer" containerID="4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1"
Sep 10 23:46:26.948413 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.948391    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:26.948782 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.948759    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:26.948835 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.948816    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:26.948869 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:26.948845    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:27.959228 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:27.958265    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" event={"ID":"ce99e366-8e62-4d15-bebf-bb9e33d501ba","Type":"ContainerStarted","Data":"31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951"}
Sep 10 23:46:27.963568 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:27.962776    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" event={"ID":"13d27ee9-2167-4cc3-84af-245a86e7672c","Type":"ContainerStarted","Data":"8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1"}
Sep 10 23:46:28.094196 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:28.094166    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:28.094446 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:28.094416    2569 patch_prober.go:28] interesting pod/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 23:46:28.094492 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:28.094469    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 23:46:28.742628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:28.742588    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:28.742838 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:28.742723    2569 patch_prober.go:28] interesting pod/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 10 23:46:28.742838 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:28.742771    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 10 23:46:29.095053 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.094455    2569 patch_prober.go:28] interesting pod/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 23:46:29.095053 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.094534    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 23:46:29.751092 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.751053    2569 patch_prober.go:28] interesting pod/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 10 23:46:29.751773 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.751365    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 10 23:46:29.791068 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.791001    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"]
Sep 10 23:46:29.803988 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.801697    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"]
Sep 10 23:46:29.806162 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.806135    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:29.809159 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.809026    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ea66ce\""
Sep 10 23:46:29.809790 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.809662    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ea66ce-dockercfg-q46tg\""
Sep 10 23:46:29.911656 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.911573    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97745\" (UniqueName: \"kubernetes.io/projected/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-kube-api-access-97745\") pod \"payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv\" (UID: \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:29.911993 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:29.911970    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv\" (UID: \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:30.012873 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.012788    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97745\" (UniqueName: \"kubernetes.io/projected/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-kube-api-access-97745\") pod \"payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv\" (UID: \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:30.012873 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.012847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv\" (UID: \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:30.013680 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.013660    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv\" (UID: \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:30.021569 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.021546    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97745\" (UniqueName: \"kubernetes.io/projected/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-kube-api-access-97745\") pod \"payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv\" (UID: \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:30.093902 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.093859    2569 patch_prober.go:28] interesting pod/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 23:46:30.094110 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.093976    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 23:46:30.125009 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.124967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:30.440043 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.439706    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"]
Sep 10 23:46:30.743543 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.743438    2569 patch_prober.go:28] interesting pod/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 10 23:46:30.743543 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.743504    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 10 23:46:30.993301 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.993258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv" event={"ID":"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394","Type":"ContainerStarted","Data":"992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55"}
Sep 10 23:46:30.993502 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.993315    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv" event={"ID":"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394","Type":"ContainerStarted","Data":"0649b35728d217e90eb88fcb357d8c4a406331e4e21874f07ea8e6981471d432"}
Sep 10 23:46:30.994011 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:30.993912    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:31.009483 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:31.009433    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv" podStartSLOduration=2.00941709 podStartE2EDuration="2.00941709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:31.007785935 +0000 UTC m=+804.661506108" watchObservedRunningTime="2026-09-10 23:46:31.00941709 +0000 UTC m=+804.663137263"
Sep 10 23:46:31.094079 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:31.094037    2569 patch_prober.go:28] interesting pod/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 23:46:31.094282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:31.094094    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 23:46:31.742499 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:31.742451    2569 patch_prober.go:28] interesting pod/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 10 23:46:31.743005 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:31.742538    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 10 23:46:32.094554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:32.094456    2569 patch_prober.go:28] interesting pod/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 23:46:32.094554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:32.094518    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 23:46:32.742832 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:32.742786    2569 patch_prober.go:28] interesting pod/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 10 23:46:32.743352 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:32.742859    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 10 23:46:33.094195 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:33.094085    2569 patch_prober.go:28] interesting pod/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 10 23:46:33.094195 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:33.094155    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 10 23:46:33.745894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:33.745866    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:33.746340 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:33.746252    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:33.747036 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:33.747005    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:46:34.097651 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:34.097567    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:34.097895 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:34.097867    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:34.098678 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:34.098653    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:39.635085 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:39.635053    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"]
Sep 10 23:46:39.635558 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:39.635502    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy" containerID="cri-o://31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951" gracePeriod=30
Sep 10 23:46:43.012441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:43.012411    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:46:44.819790 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.819761    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:44.877227 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877198    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clszl\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-kube-api-access-clszl\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877262    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-podinfo\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877291    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-socket\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877343    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-data\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877375    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-token\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877406    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-certs\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877454    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-envoy\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877481    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-credential-socket\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877514    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istiod-ca-cert\") pod \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\" (UID: \"ce99e366-8e62-4d15-bebf-bb9e33d501ba\") "
Sep 10 23:46:44.877969 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877706    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:46:44.877969 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877721    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:46:44.878070 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877972    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:44.878070 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877993    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:44.878070 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.877990    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:46:44.878203 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.878168    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:46:44.878397 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.878375    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:46:44.879984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.879952    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:46:44.880095 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.880040    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:46:44.880095 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.880065    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:46:44.880199 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.880137    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-kube-api-access-clszl" (OuterVolumeSpecName: "kube-api-access-clszl") pod "ce99e366-8e62-4d15-bebf-bb9e33d501ba" (UID: "ce99e366-8e62-4d15-bebf-bb9e33d501ba"). InnerVolumeSpecName "kube-api-access-clszl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:46:44.978747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.978667    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:44.978747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.978702    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:44.978747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.978719    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:44.978747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.978731    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:44.978747 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.978743    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce99e366-8e62-4d15-bebf-bb9e33d501ba-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:44.979076 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.978755    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce99e366-8e62-4d15-bebf-bb9e33d501ba-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:44.979076 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:44.978767    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clszl\" (UniqueName: \"kubernetes.io/projected/ce99e366-8e62-4d15-bebf-bb9e33d501ba-kube-api-access-clszl\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:46:45.085397 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.085357    2569 generic.go:356] "Generic (PLEG): container finished" podID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerID="31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951" exitCode=0
Sep 10 23:46:45.085397 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.085398    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"
Sep 10 23:46:45.085590 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.085418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" event={"ID":"ce99e366-8e62-4d15-bebf-bb9e33d501ba","Type":"ContainerDied","Data":"31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951"}
Sep 10 23:46:45.085590 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.085477    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2" event={"ID":"ce99e366-8e62-4d15-bebf-bb9e33d501ba","Type":"ContainerDied","Data":"99b16afc88033efddbc4997cf5fbcd00604b296015c99a308fcd63f36a981b52"}
Sep 10 23:46:45.085590 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.085510    2569 scope.go:117] "RemoveContainer" containerID="31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951"
Sep 10 23:46:45.094780 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.094757    2569 scope.go:117] "RemoveContainer" containerID="7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705"
Sep 10 23:46:45.100764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.100738    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"]
Sep 10 23:46:45.106371 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.106351    2569 scope.go:117] "RemoveContainer" containerID="31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951"
Sep 10 23:46:45.106747 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:46:45.106727    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951\": container with ID starting with 31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951 not found: ID does not exist" containerID="31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951"
Sep 10 23:46:45.106834 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.106755    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951"} err="failed to get container status \"31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951\": rpc error: code = NotFound desc = could not find container \"31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951\": container with ID starting with 31c6c130699e2be2fb8851e5e77a1c101c264433f6d8c6632b20a00ae6546951 not found: ID does not exist"
Sep 10 23:46:45.106834 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.106774    2569 scope.go:117] "RemoveContainer" containerID="7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705"
Sep 10 23:46:45.107111 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:46:45.107094    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705\": container with ID starting with 7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705 not found: ID does not exist" containerID="7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705"
Sep 10 23:46:45.107178 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.107123    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705"} err="failed to get container status \"7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705\": rpc error: code = NotFound desc = could not find container \"7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705\": container with ID starting with 7adc9c6874d27dc79c1106719730804b69c6ae495577944b399f811a92bf4705 not found: ID does not exist"
Sep 10 23:46:45.108100 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:45.108082    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9dfc47f0-openshift-default-5648596546-bdjx2"]
Sep 10 23:46:46.093894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:46.093797    2569 generic.go:356] "Generic (PLEG): container finished" podID="91d38d48-20f1-4730-887c-1457b6849524" containerID="52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd" exitCode=0
Sep 10 23:46:46.093894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:46.093858    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" event={"ID":"91d38d48-20f1-4730-887c-1457b6849524","Type":"ContainerDied","Data":"52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd"}
Sep 10 23:46:46.094494 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:46.094377    2569 scope.go:117] "RemoveContainer" containerID="52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd"
Sep 10 23:46:46.095495 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:46.095465    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:46.095598 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:46.095575    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:46.095663 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:46.095616    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:46:46.982532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:46.982490    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" path="/var/lib/kubelet/pods/ce99e366-8e62-4d15-bebf-bb9e33d501ba/volumes"
Sep 10 23:46:47.101264 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.101228    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" event={"ID":"91d38d48-20f1-4730-887c-1457b6849524","Type":"ContainerStarted","Data":"06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da"}
Sep 10 23:46:47.506569 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.506529    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"]
Sep 10 23:46:47.507429 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.507400    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy"
Sep 10 23:46:47.507429 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.507430    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy"
Sep 10 23:46:47.507607 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.507543    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy"
Sep 10 23:46:47.507607 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.507559    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy"
Sep 10 23:46:47.511277 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.511257    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:47.513566 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.513546    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-cd653317\""
Sep 10 23:46:47.513687 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.513656    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-cd653317-dockercfg-sn5mv\""
Sep 10 23:46:47.518664 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.518641    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"]
Sep 10 23:46:47.520167 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.520151    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:47.520399 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.520371    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 10 23:46:47.520523 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.520420    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 10 23:46:47.605983 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.605925    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/407e65b9-4eb5-4190-b240-0b667a94ea5f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s\" (UID: \"407e65b9-4eb5-4190-b240-0b667a94ea5f\") " pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:47.606185 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.606019    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f25z\" (UniqueName: \"kubernetes.io/projected/407e65b9-4eb5-4190-b240-0b667a94ea5f-kube-api-access-9f25z\") pod \"payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s\" (UID: \"407e65b9-4eb5-4190-b240-0b667a94ea5f\") " pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:47.707297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.707262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/407e65b9-4eb5-4190-b240-0b667a94ea5f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s\" (UID: \"407e65b9-4eb5-4190-b240-0b667a94ea5f\") " pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:47.707297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.707311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f25z\" (UniqueName: \"kubernetes.io/projected/407e65b9-4eb5-4190-b240-0b667a94ea5f-kube-api-access-9f25z\") pod \"payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s\" (UID: \"407e65b9-4eb5-4190-b240-0b667a94ea5f\") " pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:47.708042 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.708021    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/407e65b9-4eb5-4190-b240-0b667a94ea5f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s\" (UID: \"407e65b9-4eb5-4190-b240-0b667a94ea5f\") " pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:47.714337 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.714305    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f25z\" (UniqueName: \"kubernetes.io/projected/407e65b9-4eb5-4190-b240-0b667a94ea5f-kube-api-access-9f25z\") pod \"payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s\" (UID: \"407e65b9-4eb5-4190-b240-0b667a94ea5f\") " pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:47.824105 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:47.824025    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:48.122823 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:48.122802    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"]
Sep 10 23:46:48.124668 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:46:48.124640    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod407e65b9_4eb5_4190_b240_0b667a94ea5f.slice/crio-bc4b3f0926a31dfbc8a620007968ab6de332dd521ca620bd3b18ed14a02f16b6 WatchSource:0}: Error finding container bc4b3f0926a31dfbc8a620007968ab6de332dd521ca620bd3b18ed14a02f16b6: Status 404 returned error can't find the container with id bc4b3f0926a31dfbc8a620007968ab6de332dd521ca620bd3b18ed14a02f16b6
Sep 10 23:46:48.520599 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:48.520503    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 10 23:46:48.520599 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:48.520555    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 10 23:46:49.113837 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:49.113801    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s" event={"ID":"407e65b9-4eb5-4190-b240-0b667a94ea5f","Type":"ContainerStarted","Data":"53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e"}
Sep 10 23:46:49.114066 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:49.113861    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s" event={"ID":"407e65b9-4eb5-4190-b240-0b667a94ea5f","Type":"ContainerStarted","Data":"bc4b3f0926a31dfbc8a620007968ab6de332dd521ca620bd3b18ed14a02f16b6"}
Sep 10 23:46:49.114066 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:49.113979    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:46:49.130568 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:49.130522    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s" podStartSLOduration=2.13050984 podStartE2EDuration="2.13050984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:46:49.128495448 +0000 UTC m=+822.782215616" watchObservedRunningTime="2026-09-10 23:46:49.13050984 +0000 UTC m=+822.784230055"
Sep 10 23:46:49.521072 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:49.520982    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 10 23:46:49.521072 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:49.521036    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 10 23:46:50.520800 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:50.520758    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 10 23:46:50.521318 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:50.520823    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 10 23:46:51.521260 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:51.521222    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Sep 10 23:46:51.521659 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:51.521290    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Sep 10 23:46:52.526236 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:52.526203    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:52.526738 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:52.526492    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:46:52.527177 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:46:52.527157    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:47:00.121779 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:00.121748    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:47:18.362328 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.362291    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"]
Sep 10 23:47:18.362926 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.362799    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s" podUID="407e65b9-4eb5-4190-b240-0b667a94ea5f" containerName="payload-processing" containerID="cri-o://53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e" gracePeriod=30
Sep 10 23:47:18.640104 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.640078    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:47:18.732109 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.732078    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/407e65b9-4eb5-4190-b240-0b667a94ea5f-plugins-config-volume\") pod \"407e65b9-4eb5-4190-b240-0b667a94ea5f\" (UID: \"407e65b9-4eb5-4190-b240-0b667a94ea5f\") "
Sep 10 23:47:18.732306 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.732154    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9f25z\" (UniqueName: \"kubernetes.io/projected/407e65b9-4eb5-4190-b240-0b667a94ea5f-kube-api-access-9f25z\") pod \"407e65b9-4eb5-4190-b240-0b667a94ea5f\" (UID: \"407e65b9-4eb5-4190-b240-0b667a94ea5f\") "
Sep 10 23:47:18.732512 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.732486    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/407e65b9-4eb5-4190-b240-0b667a94ea5f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "407e65b9-4eb5-4190-b240-0b667a94ea5f" (UID: "407e65b9-4eb5-4190-b240-0b667a94ea5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:47:18.734219 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.734193    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/407e65b9-4eb5-4190-b240-0b667a94ea5f-kube-api-access-9f25z" (OuterVolumeSpecName: "kube-api-access-9f25z") pod "407e65b9-4eb5-4190-b240-0b667a94ea5f" (UID: "407e65b9-4eb5-4190-b240-0b667a94ea5f"). InnerVolumeSpecName "kube-api-access-9f25z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:47:18.833883 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.833849    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/407e65b9-4eb5-4190-b240-0b667a94ea5f-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:47:18.833883 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:18.833881    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9f25z\" (UniqueName: \"kubernetes.io/projected/407e65b9-4eb5-4190-b240-0b667a94ea5f-kube-api-access-9f25z\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:47:19.281430 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.281336    2569 generic.go:356] "Generic (PLEG): container finished" podID="407e65b9-4eb5-4190-b240-0b667a94ea5f" containerID="53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e" exitCode=0
Sep 10 23:47:19.281430 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.281376    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"
Sep 10 23:47:19.281430 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.281411    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s" event={"ID":"407e65b9-4eb5-4190-b240-0b667a94ea5f","Type":"ContainerDied","Data":"53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e"}
Sep 10 23:47:19.281679 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.281462    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s" event={"ID":"407e65b9-4eb5-4190-b240-0b667a94ea5f","Type":"ContainerDied","Data":"bc4b3f0926a31dfbc8a620007968ab6de332dd521ca620bd3b18ed14a02f16b6"}
Sep 10 23:47:19.281679 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.281483    2569 scope.go:117] "RemoveContainer" containerID="53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e"
Sep 10 23:47:19.290613 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.290596    2569 scope.go:117] "RemoveContainer" containerID="53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e"
Sep 10 23:47:19.290855 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:47:19.290840    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e\": container with ID starting with 53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e not found: ID does not exist" containerID="53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e"
Sep 10 23:47:19.290902 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.290861    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e"} err="failed to get container status \"53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e\": rpc error: code = NotFound desc = could not find container \"53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e\": container with ID starting with 53e9c8bf946533436eae7fd28d56731cd53990bd4da6e67b7857d613ef496c8e not found: ID does not exist"
Sep 10 23:47:19.296280 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.296259    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"]
Sep 10 23:47:19.300004 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.299981    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-cd653317-5d84b4469d-4nk5s"]
Sep 10 23:47:19.409512 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.409476    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"]
Sep 10 23:47:19.410200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.410173    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="407e65b9-4eb5-4190-b240-0b667a94ea5f" containerName="payload-processing"
Sep 10 23:47:19.410326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.410202    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="407e65b9-4eb5-4190-b240-0b667a94ea5f" containerName="payload-processing"
Sep 10 23:47:19.410326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.410234    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy"
Sep 10 23:47:19.410326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.410243    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce99e366-8e62-4d15-bebf-bb9e33d501ba" containerName="istio-proxy"
Sep 10 23:47:19.410480 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.410362    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="407e65b9-4eb5-4190-b240-0b667a94ea5f" containerName="payload-processing"
Sep 10 23:47:19.414501 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.414472    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.416398 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.416377    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d949bf5d-openshift-default-dockercfg-7wh5c\""
Sep 10 23:47:19.426149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.426122    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"]
Sep 10 23:47:19.542906 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.542797    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjf44\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-kube-api-access-tjf44\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.542906 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.542843    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0a83af-0042-4ddb-9d16-608ace65a13a-istiod-ca-cert\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.543119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.542949    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-socket\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.543119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.542982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-token\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.543119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.543003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-data\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.543119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.543063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-podinfo\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.543119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.543091    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-envoy\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.543119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.543113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-credential-socket\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.543311 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.543140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-certs\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644158 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644127    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjf44\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-kube-api-access-tjf44\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644158 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644162    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0a83af-0042-4ddb-9d16-608ace65a13a-istiod-ca-cert\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644191    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-socket\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644208    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-token\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-data\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644269    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-podinfo\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-envoy\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-credential-socket\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644343    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-certs\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644775 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644678    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-socket\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644775 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-credential-socket\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644876 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644770    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-data\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644967 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644873    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0a83af-0042-4ddb-9d16-608ace65a13a-istiod-ca-cert\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.644967 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.644888    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-certs\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.646594 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.646572    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-envoy\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.646801 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.646782    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-podinfo\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.651657 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.651630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-token\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.651773 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.651749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjf44\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-kube-api-access-tjf44\") pod \"e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") " pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:19.727000 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:19.726963    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:20.001957 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.001919    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"]
Sep 10 23:47:20.004118 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:47:20.004089    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0a83af_0042_4ddb_9d16_608ace65a13a.slice/crio-f1ddb01dc040379560a37d115402ee3c693603021236cd883e1c9f23b9f9ce07 WatchSource:0}: Error finding container f1ddb01dc040379560a37d115402ee3c693603021236cd883e1c9f23b9f9ce07: Status 404 returned error can't find the container with id f1ddb01dc040379560a37d115402ee3c693603021236cd883e1c9f23b9f9ce07
Sep 10 23:47:20.006463 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.006430    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:47:20.006542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.006497    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:47:20.006542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.006531    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:47:20.291338 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.291247    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" event={"ID":"0f0a83af-0042-4ddb-9d16-608ace65a13a","Type":"ContainerStarted","Data":"279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a"}
Sep 10 23:47:20.291338 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.291310    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" event={"ID":"0f0a83af-0042-4ddb-9d16-608ace65a13a","Type":"ContainerStarted","Data":"f1ddb01dc040379560a37d115402ee3c693603021236cd883e1c9f23b9f9ce07"}
Sep 10 23:47:20.309290 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.309225    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" podStartSLOduration=1.309204756 podStartE2EDuration="1.309204756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:47:20.308181128 +0000 UTC m=+853.961901301" watchObservedRunningTime="2026-09-10 23:47:20.309204756 +0000 UTC m=+853.962924931"
Sep 10 23:47:20.727326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.727294    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:20.979436 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:20.979350    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="407e65b9-4eb5-4190-b240-0b667a94ea5f" path="/var/lib/kubelet/pods/407e65b9-4eb5-4190-b240-0b667a94ea5f/volumes"
Sep 10 23:47:21.732947 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:21.732912    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:22.302821 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:22.302789    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:22.303919 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:22.303898    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:42.415402 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:42.415369    2569 generic.go:356] "Generic (PLEG): container finished" podID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerID="279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a" exitCode=0
Sep 10 23:47:42.415847 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:42.415429    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" event={"ID":"0f0a83af-0042-4ddb-9d16-608ace65a13a","Type":"ContainerDied","Data":"279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a"}
Sep 10 23:47:42.415847 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:42.415839    2569 scope.go:117] "RemoveContainer" containerID="279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a"
Sep 10 23:47:42.416973 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:42.416910    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:47:42.417103 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:42.417020    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:47:42.417103 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:42.417070    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:47:43.422299 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:43.422261    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" event={"ID":"0f0a83af-0042-4ddb-9d16-608ace65a13a","Type":"ContainerStarted","Data":"6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e"}
Sep 10 23:47:43.727336 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:43.727241    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:43.727516 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:43.727490    2569 patch_prober.go:28] interesting pod/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 10 23:47:43.727574 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:43.727551    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 10 23:47:44.728134 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:44.728091    2569 patch_prober.go:28] interesting pod/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 10 23:47:44.728542 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:44.728196    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 10 23:47:45.727537 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:45.727497    2569 patch_prober.go:28] interesting pod/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 10 23:47:45.727726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:45.727566    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 10 23:47:46.728246 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:46.728210    2569 patch_prober.go:28] interesting pod/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 10 23:47:46.728633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:46.728267    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 10 23:47:47.727781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:47.727743    2569 patch_prober.go:28] interesting pod/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Sep 10 23:47:47.727991 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:47.727800    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Sep 10 23:47:48.741722 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:48.741689    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:48.742217 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:48.742073    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:48.742809 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:48.742789    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:54.568967 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:54.568905    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"]
Sep 10 23:47:54.569516 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:54.569149    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy" containerID="cri-o://6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e" gracePeriod=30
Sep 10 23:47:58.792588 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:58.792553    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"]
Sep 10 23:47:58.796423 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:58.796401    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"
Sep 10 23:47:58.798453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:58.798434    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:47:58.798453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:58.798444    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:47:58.802668 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:58.802644    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"]
Sep 10 23:47:58.947846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:58.947795    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e688402-2fed-4f28-8bdb-e5865cecdcd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-abe3de-kjtcv\" (UID: \"4e688402-2fed-4f28-8bdb-e5865cecdcd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"
Sep 10 23:47:59.048965 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.048821    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e688402-2fed-4f28-8bdb-e5865cecdcd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-abe3de-kjtcv\" (UID: \"4e688402-2fed-4f28-8bdb-e5865cecdcd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"
Sep 10 23:47:59.049487 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.049466    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e688402-2fed-4f28-8bdb-e5865cecdcd7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-abe3de-kjtcv\" (UID: \"4e688402-2fed-4f28-8bdb-e5865cecdcd7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"
Sep 10 23:47:59.146829 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.146794    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"
Sep 10 23:47:59.415416 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.415391    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"]
Sep 10 23:47:59.417874 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:47:59.417840    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e688402_2fed_4f28_8bdb_e5865cecdcd7.slice/crio-d98f6fe2793df1997770e4d3f31a47e597119bb30bd15c076517adfcd1bb7ee6 WatchSource:0}: Error finding container d98f6fe2793df1997770e4d3f31a47e597119bb30bd15c076517adfcd1bb7ee6: Status 404 returned error can't find the container with id d98f6fe2793df1997770e4d3f31a47e597119bb30bd15c076517adfcd1bb7ee6
Sep 10 23:47:59.514399 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.514365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv" event={"ID":"4e688402-2fed-4f28-8bdb-e5865cecdcd7","Type":"ContainerStarted","Data":"d98f6fe2793df1997770e4d3f31a47e597119bb30bd15c076517adfcd1bb7ee6"}
Sep 10 23:47:59.824731 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.824705    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:47:59.957124 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957027    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0a83af-0042-4ddb-9d16-608ace65a13a-istiod-ca-cert\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957124 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957121    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-envoy\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957349 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957158    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-certs\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957349 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957191    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-data\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957349 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957222    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-socket\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957349 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957265    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-credential-socket\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957349 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957306    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-podinfo\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957591 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957367    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-token\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957591 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957424    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjf44\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-kube-api-access-tjf44\") pod \"0f0a83af-0042-4ddb-9d16-608ace65a13a\" (UID: \"0f0a83af-0042-4ddb-9d16-608ace65a13a\") "
Sep 10 23:47:59.957591 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957426    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f0a83af-0042-4ddb-9d16-608ace65a13a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:47:59.957746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957676    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:47:59.957746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957697    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:47:59.957852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957785    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f0a83af-0042-4ddb-9d16-608ace65a13a-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:47:59.957852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957803    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:47:59.957852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957815    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:47:59.958040 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.957890    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:47:59.958111 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.958084    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:47:59.960024 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.959915    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-kube-api-access-tjf44" (OuterVolumeSpecName: "kube-api-access-tjf44") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "kube-api-access-tjf44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:47:59.960619 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.960340    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:47:59.960729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.960666    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:47:59.960789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:47:59.960718    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f0a83af-0042-4ddb-9d16-608ace65a13a" (UID: "0f0a83af-0042-4ddb-9d16-608ace65a13a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:48:00.058684 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.058649    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:00.058684 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.058680    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:00.058684 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.058689    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f0a83af-0042-4ddb-9d16-608ace65a13a-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:00.059068 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.058698    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:00.059068 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.058706    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:00.059068 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.058714    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjf44\" (UniqueName: \"kubernetes.io/projected/0f0a83af-0042-4ddb-9d16-608ace65a13a-kube-api-access-tjf44\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:00.232533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.232456    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"]
Sep 10 23:48:00.232881 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.232868    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy"
Sep 10 23:48:00.232930 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.232883    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy"
Sep 10 23:48:00.232930 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.232893    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy"
Sep 10 23:48:00.232930 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.232898    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy"
Sep 10 23:48:00.233044 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.232988    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy"
Sep 10 23:48:00.233044 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.233000    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerName="istio-proxy"
Sep 10 23:48:00.236205 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.236189    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:00.238253 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.238228    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-51b15052-dockercfg-wjvjs\""
Sep 10 23:48:00.238398 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.238258    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-51b15052\""
Sep 10 23:48:00.246052 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.246015    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"]
Sep 10 23:48:00.361847 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.361818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c82r\" (UniqueName: \"kubernetes.io/projected/2942ec2a-8356-4417-8eea-0c2477a7bee3-kube-api-access-4c82r\") pod \"payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg\" (UID: \"2942ec2a-8356-4417-8eea-0c2477a7bee3\") " pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:00.362000 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.361930    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2942ec2a-8356-4417-8eea-0c2477a7bee3-plugins-config-volume\") pod \"payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg\" (UID: \"2942ec2a-8356-4417-8eea-0c2477a7bee3\") " pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:00.463132 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.463097    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c82r\" (UniqueName: \"kubernetes.io/projected/2942ec2a-8356-4417-8eea-0c2477a7bee3-kube-api-access-4c82r\") pod \"payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg\" (UID: \"2942ec2a-8356-4417-8eea-0c2477a7bee3\") " pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:00.463298 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.463164    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2942ec2a-8356-4417-8eea-0c2477a7bee3-plugins-config-volume\") pod \"payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg\" (UID: \"2942ec2a-8356-4417-8eea-0c2477a7bee3\") " pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:00.463804 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.463786    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2942ec2a-8356-4417-8eea-0c2477a7bee3-plugins-config-volume\") pod \"payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg\" (UID: \"2942ec2a-8356-4417-8eea-0c2477a7bee3\") " pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:00.470528 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.470507    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c82r\" (UniqueName: \"kubernetes.io/projected/2942ec2a-8356-4417-8eea-0c2477a7bee3-kube-api-access-4c82r\") pod \"payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg\" (UID: \"2942ec2a-8356-4417-8eea-0c2477a7bee3\") " pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:00.522007 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.521900    2569 generic.go:356] "Generic (PLEG): container finished" podID="4e688402-2fed-4f28-8bdb-e5865cecdcd7" containerID="8c5f363691845129684bdb2f2149e5d939e3a3e43d4a2a41157e40b7a1c332cf" exitCode=0
Sep 10 23:48:00.522150 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.521993    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv" event={"ID":"4e688402-2fed-4f28-8bdb-e5865cecdcd7","Type":"ContainerDied","Data":"8c5f363691845129684bdb2f2149e5d939e3a3e43d4a2a41157e40b7a1c332cf"}
Sep 10 23:48:00.523538 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.523510    2569 generic.go:356] "Generic (PLEG): container finished" podID="0f0a83af-0042-4ddb-9d16-608ace65a13a" containerID="6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e" exitCode=0
Sep 10 23:48:00.523660 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.523538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" event={"ID":"0f0a83af-0042-4ddb-9d16-608ace65a13a","Type":"ContainerDied","Data":"6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e"}
Sep 10 23:48:00.523660 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.523549    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"
Sep 10 23:48:00.523660 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.523583    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp" event={"ID":"0f0a83af-0042-4ddb-9d16-608ace65a13a","Type":"ContainerDied","Data":"f1ddb01dc040379560a37d115402ee3c693603021236cd883e1c9f23b9f9ce07"}
Sep 10 23:48:00.523660 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.523605    2569 scope.go:117] "RemoveContainer" containerID="6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e"
Sep 10 23:48:00.532846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.532833    2569 scope.go:117] "RemoveContainer" containerID="279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a"
Sep 10 23:48:00.541763 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.541746    2569 scope.go:117] "RemoveContainer" containerID="6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e"
Sep 10 23:48:00.542058 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:48:00.542039    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e\": container with ID starting with 6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e not found: ID does not exist" containerID="6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e"
Sep 10 23:48:00.542127 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.542069    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e"} err="failed to get container status \"6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e\": rpc error: code = NotFound desc = could not find container \"6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e\": container with ID starting with 6d8f185893acee32e1c3022c62db56cf6a47eb111d20a2e22dcd11de3b8ce39e not found: ID does not exist"
Sep 10 23:48:00.542127 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.542086    2569 scope.go:117] "RemoveContainer" containerID="279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a"
Sep 10 23:48:00.542349 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:48:00.542331    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a\": container with ID starting with 279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a not found: ID does not exist" containerID="279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a"
Sep 10 23:48:00.542403 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.542358    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a"} err="failed to get container status \"279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a\": rpc error: code = NotFound desc = could not find container \"279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a\": container with ID starting with 279f361793302d8a27f4e95a547b2699825f1dfe84c1a57afdd85a4c70a37c3a not found: ID does not exist"
Sep 10 23:48:00.547374 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.547353    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"]
Sep 10 23:48:00.549666 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.549644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:00.550828 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.550805    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d949bf5d-openshift-default-7b4fbb946b-2bqkp"]
Sep 10 23:48:00.823710 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.823684    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"]
Sep 10 23:48:00.825820 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:48:00.825791    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2942ec2a_8356_4417_8eea_0c2477a7bee3.slice/crio-13dadba4e607653fbe0b6e7dbbdfbf518fa8ac2dd71d33bb8f1469d811221717 WatchSource:0}: Error finding container 13dadba4e607653fbe0b6e7dbbdfbf518fa8ac2dd71d33bb8f1469d811221717: Status 404 returned error can't find the container with id 13dadba4e607653fbe0b6e7dbbdfbf518fa8ac2dd71d33bb8f1469d811221717
Sep 10 23:48:00.982287 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:00.982250    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f0a83af-0042-4ddb-9d16-608ace65a13a" path="/var/lib/kubelet/pods/0f0a83af-0042-4ddb-9d16-608ace65a13a/volumes"
Sep 10 23:48:01.531994 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:01.531951    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg" event={"ID":"2942ec2a-8356-4417-8eea-0c2477a7bee3","Type":"ContainerStarted","Data":"12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00"}
Sep 10 23:48:01.531994 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:01.532021    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:01.532266 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:01.532046    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg" event={"ID":"2942ec2a-8356-4417-8eea-0c2477a7bee3","Type":"ContainerStarted","Data":"13dadba4e607653fbe0b6e7dbbdfbf518fa8ac2dd71d33bb8f1469d811221717"}
Sep 10 23:48:01.551792 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:01.551731    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg" podStartSLOduration=1.551715117 podStartE2EDuration="1.551715117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:48:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:48:01.549760815 +0000 UTC m=+895.203481001" watchObservedRunningTime="2026-09-10 23:48:01.551715117 +0000 UTC m=+895.205435322"
Sep 10 23:48:01.730077 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:01.730028    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"
Sep 10 23:48:01.875965 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:01.875916    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e688402-2fed-4f28-8bdb-e5865cecdcd7-maas-api-service-ca\") pod \"4e688402-2fed-4f28-8bdb-e5865cecdcd7\" (UID: \"4e688402-2fed-4f28-8bdb-e5865cecdcd7\") "
Sep 10 23:48:01.876561 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:01.876264    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e688402-2fed-4f28-8bdb-e5865cecdcd7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e688402-2fed-4f28-8bdb-e5865cecdcd7" (UID: "4e688402-2fed-4f28-8bdb-e5865cecdcd7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:48:01.877007 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:01.876981    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e688402-2fed-4f28-8bdb-e5865cecdcd7-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:02.541329 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:02.541290    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv" event={"ID":"4e688402-2fed-4f28-8bdb-e5865cecdcd7","Type":"ContainerDied","Data":"d98f6fe2793df1997770e4d3f31a47e597119bb30bd15c076517adfcd1bb7ee6"}
Sep 10 23:48:02.541329 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:02.541319    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"
Sep 10 23:48:02.541536 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:02.541352    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d98f6fe2793df1997770e4d3f31a47e597119bb30bd15c076517adfcd1bb7ee6"
Sep 10 23:48:07.020272 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:07.020240    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:48:07.021618 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:07.021584    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:48:07.029054 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:07.029033    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:48:07.030273 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:07.030256    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:48:09.567978 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:09.567946    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"]
Sep 10 23:48:09.568447 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:09.568170    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd" podUID="02198fef-fa7e-485a-954f-93d599cc85a3" containerName="payload-processing" containerID="cri-o://c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec" gracePeriod=30
Sep 10 23:48:09.575272 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:09.575233    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"]
Sep 10 23:48:09.576052 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:09.575642    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn" podUID="99d1ec48-7e67-4039-b33b-21b3372b5c2c" containerName="payload-pre-processing" containerID="cri-o://4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976" gracePeriod=30
Sep 10 23:48:09.728715 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:09.728646    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"]
Sep 10 23:48:09.730819 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:09.730787    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-abe3de-kjtcv"]
Sep 10 23:48:09.871097 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:09.871074    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:48:09.910086 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:09.910058    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:48:10.068903 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.068868    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbbtn\" (UniqueName: \"kubernetes.io/projected/99d1ec48-7e67-4039-b33b-21b3372b5c2c-kube-api-access-wbbtn\") pod \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\" (UID: \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\") "
Sep 10 23:48:10.069090 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.068970    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02198fef-fa7e-485a-954f-93d599cc85a3-plugins-config-volume\") pod \"02198fef-fa7e-485a-954f-93d599cc85a3\" (UID: \"02198fef-fa7e-485a-954f-93d599cc85a3\") "
Sep 10 23:48:10.069090 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.069016    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnbxw\" (UniqueName: \"kubernetes.io/projected/02198fef-fa7e-485a-954f-93d599cc85a3-kube-api-access-cnbxw\") pod \"02198fef-fa7e-485a-954f-93d599cc85a3\" (UID: \"02198fef-fa7e-485a-954f-93d599cc85a3\") "
Sep 10 23:48:10.069090 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.069082    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99d1ec48-7e67-4039-b33b-21b3372b5c2c-plugins-config-volume\") pod \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\" (UID: \"99d1ec48-7e67-4039-b33b-21b3372b5c2c\") "
Sep 10 23:48:10.069443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.069410    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02198fef-fa7e-485a-954f-93d599cc85a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02198fef-fa7e-485a-954f-93d599cc85a3" (UID: "02198fef-fa7e-485a-954f-93d599cc85a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:48:10.069545 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.069478    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d1ec48-7e67-4039-b33b-21b3372b5c2c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99d1ec48-7e67-4039-b33b-21b3372b5c2c" (UID: "99d1ec48-7e67-4039-b33b-21b3372b5c2c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:48:10.071064 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.071037    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02198fef-fa7e-485a-954f-93d599cc85a3-kube-api-access-cnbxw" (OuterVolumeSpecName: "kube-api-access-cnbxw") pod "02198fef-fa7e-485a-954f-93d599cc85a3" (UID: "02198fef-fa7e-485a-954f-93d599cc85a3"). InnerVolumeSpecName "kube-api-access-cnbxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:48:10.071146 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.071096    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d1ec48-7e67-4039-b33b-21b3372b5c2c-kube-api-access-wbbtn" (OuterVolumeSpecName: "kube-api-access-wbbtn") pod "99d1ec48-7e67-4039-b33b-21b3372b5c2c" (UID: "99d1ec48-7e67-4039-b33b-21b3372b5c2c"). InnerVolumeSpecName "kube-api-access-wbbtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:48:10.170205 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.170171    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99d1ec48-7e67-4039-b33b-21b3372b5c2c-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:10.170205 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.170197    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbbtn\" (UniqueName: \"kubernetes.io/projected/99d1ec48-7e67-4039-b33b-21b3372b5c2c-kube-api-access-wbbtn\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:10.170205 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.170207    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02198fef-fa7e-485a-954f-93d599cc85a3-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:10.170205 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.170217    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnbxw\" (UniqueName: \"kubernetes.io/projected/02198fef-fa7e-485a-954f-93d599cc85a3-kube-api-access-cnbxw\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:10.588549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.588514    2569 generic.go:356] "Generic (PLEG): container finished" podID="99d1ec48-7e67-4039-b33b-21b3372b5c2c" containerID="4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976" exitCode=0
Sep 10 23:48:10.588985 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.588590    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"
Sep 10 23:48:10.588985 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.588700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn" event={"ID":"99d1ec48-7e67-4039-b33b-21b3372b5c2c","Type":"ContainerDied","Data":"4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976"}
Sep 10 23:48:10.588985 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.588750    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn" event={"ID":"99d1ec48-7e67-4039-b33b-21b3372b5c2c","Type":"ContainerDied","Data":"7640dcd0d846cc1064d7060b28176aea59d5421d8c221a384fbcd94bd564babe"}
Sep 10 23:48:10.588985 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.588773    2569 scope.go:117] "RemoveContainer" containerID="4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976"
Sep 10 23:48:10.590467 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.590441    2569 generic.go:356] "Generic (PLEG): container finished" podID="02198fef-fa7e-485a-954f-93d599cc85a3" containerID="c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec" exitCode=0
Sep 10 23:48:10.590635 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.590490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd" event={"ID":"02198fef-fa7e-485a-954f-93d599cc85a3","Type":"ContainerDied","Data":"c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec"}
Sep 10 23:48:10.590635 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.590502    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"
Sep 10 23:48:10.590635 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.590530    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd" event={"ID":"02198fef-fa7e-485a-954f-93d599cc85a3","Type":"ContainerDied","Data":"0725a7b00c6cbb66746352e28df08fc1aae5b90a998261d2ffbcb4033e93b58a"}
Sep 10 23:48:10.598671 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.598654    2569 scope.go:117] "RemoveContainer" containerID="4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976"
Sep 10 23:48:10.599006 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:48:10.598988    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976\": container with ID starting with 4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976 not found: ID does not exist" containerID="4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976"
Sep 10 23:48:10.599099 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.599016    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976"} err="failed to get container status \"4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976\": rpc error: code = NotFound desc = could not find container \"4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976\": container with ID starting with 4b014c506aef722ba089ad8bffefdda1f5d080c7d01b6d3e8b511ac2cd1de976 not found: ID does not exist"
Sep 10 23:48:10.599099 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.599039    2569 scope.go:117] "RemoveContainer" containerID="c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec"
Sep 10 23:48:10.607206 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.607182    2569 scope.go:117] "RemoveContainer" containerID="c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec"
Sep 10 23:48:10.607450 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:48:10.607433    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec\": container with ID starting with c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec not found: ID does not exist" containerID="c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec"
Sep 10 23:48:10.607513 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.607456    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec"} err="failed to get container status \"c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec\": rpc error: code = NotFound desc = could not find container \"c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec\": container with ID starting with c403aa63674c43f7230393fcefb87b6e7d87f2f95aeb3d9066fec2f0586c81ec not found: ID does not exist"
Sep 10 23:48:10.613284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.613261    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"]
Sep 10 23:48:10.619906 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.619885    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-abe3de-58cd79478b-wwxzn"]
Sep 10 23:48:10.632020 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.632001    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"]
Sep 10 23:48:10.640452 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.640427    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-abe3de-77fc6f968f-7lcbd"]
Sep 10 23:48:10.979369 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.979339    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02198fef-fa7e-485a-954f-93d599cc85a3" path="/var/lib/kubelet/pods/02198fef-fa7e-485a-954f-93d599cc85a3/volumes"
Sep 10 23:48:10.979695 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.979683    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e688402-2fed-4f28-8bdb-e5865cecdcd7" path="/var/lib/kubelet/pods/4e688402-2fed-4f28-8bdb-e5865cecdcd7/volumes"
Sep 10 23:48:10.980019 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:10.980006    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d1ec48-7e67-4039-b33b-21b3372b5c2c" path="/var/lib/kubelet/pods/99d1ec48-7e67-4039-b33b-21b3372b5c2c/volumes"
Sep 10 23:48:12.543804 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:12.543775    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:16.149515 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.149476    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"]
Sep 10 23:48:16.150005 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.149728    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy" containerID="cri-o://8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1" gracePeriod=30
Sep 10 23:48:16.874741 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.874708    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"]
Sep 10 23:48:16.875197 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875179    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e688402-2fed-4f28-8bdb-e5865cecdcd7" containerName="revoke-keys"
Sep 10 23:48:16.875300 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875199    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e688402-2fed-4f28-8bdb-e5865cecdcd7" containerName="revoke-keys"
Sep 10 23:48:16.875300 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875222    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d1ec48-7e67-4039-b33b-21b3372b5c2c" containerName="payload-pre-processing"
Sep 10 23:48:16.875300 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875231    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d1ec48-7e67-4039-b33b-21b3372b5c2c" containerName="payload-pre-processing"
Sep 10 23:48:16.875300 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875263    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02198fef-fa7e-485a-954f-93d599cc85a3" containerName="payload-processing"
Sep 10 23:48:16.875300 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875271    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="02198fef-fa7e-485a-954f-93d599cc85a3" containerName="payload-processing"
Sep 10 23:48:16.875548 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875371    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e688402-2fed-4f28-8bdb-e5865cecdcd7" containerName="revoke-keys"
Sep 10 23:48:16.875548 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875388    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="02198fef-fa7e-485a-954f-93d599cc85a3" containerName="payload-processing"
Sep 10 23:48:16.875548 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.875400    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d1ec48-7e67-4039-b33b-21b3372b5c2c" containerName="payload-pre-processing"
Sep 10 23:48:16.880102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.880081    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.882211 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.882188    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-a66ef6-openshift-default-dockercfg-hgfrk\""
Sep 10 23:48:16.891726 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.891702    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"]
Sep 10 23:48:16.926665 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-socket\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.926665 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926668    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-data\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.926966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-istio-token\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.926966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-certs\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.926966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f77f243a-285b-4ab8-a85e-276079d16d79-istiod-ca-cert\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.926966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-envoy\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.926966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f77f243a-285b-4ab8-a85e-276079d16d79-istio-podinfo\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.926966 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926862    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkdnw\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-kube-api-access-gkdnw\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:16.927184 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:16.926977    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-credential-socket\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.027683 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.027649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-envoy\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.027683 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.027682    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f77f243a-285b-4ab8-a85e-276079d16d79-istio-podinfo\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.027968 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.027703    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkdnw\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-kube-api-access-gkdnw\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.027968 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.027838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-credential-socket\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.027968 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.027905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-socket\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.027968 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.027945    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-data\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.028189 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.027991    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-istio-token\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.028189 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.028146    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-certs\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.028299 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.028211    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f77f243a-285b-4ab8-a85e-276079d16d79-istiod-ca-cert\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.028299 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.028227    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-credential-socket\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.028299 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.028281    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-socket\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.028575 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.028551    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-data\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.028808 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.028786    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-certs\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.028901 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.028884    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f77f243a-285b-4ab8-a85e-276079d16d79-istiod-ca-cert\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.030035 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.030012    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-envoy\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.030416 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.030393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f77f243a-285b-4ab8-a85e-276079d16d79-istio-podinfo\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.035430 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.035406    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-istio-token\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.035543 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.035485    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkdnw\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-kube-api-access-gkdnw\") pod \"e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") " pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.193499 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.193402    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:17.495801 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.495768    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"]
Sep 10 23:48:17.497042 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:48:17.497010    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf77f243a_285b_4ab8_a85e_276079d16d79.slice/crio-b2bd3f8dc714e287f1a8a99e0fa03562b076c0749bd7c7d7eae3e57c2448afad WatchSource:0}: Error finding container b2bd3f8dc714e287f1a8a99e0fa03562b076c0749bd7c7d7eae3e57c2448afad: Status 404 returned error can't find the container with id b2bd3f8dc714e287f1a8a99e0fa03562b076c0749bd7c7d7eae3e57c2448afad
Sep 10 23:48:17.499537 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.499504    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:48:17.499615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.499567    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:48:17.499615 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.499595    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:48:17.636645 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.636589    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" event={"ID":"f77f243a-285b-4ab8-a85e-276079d16d79","Type":"ContainerStarted","Data":"cb55b8bac0a46cafdafe6dc8fdffca8fdde9b9002657dd954a5c1ff0e916ca0f"}
Sep 10 23:48:17.636811 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.636675    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" event={"ID":"f77f243a-285b-4ab8-a85e-276079d16d79","Type":"ContainerStarted","Data":"b2bd3f8dc714e287f1a8a99e0fa03562b076c0749bd7c7d7eae3e57c2448afad"}
Sep 10 23:48:17.658152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:17.658098    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" podStartSLOduration=1.658081836 podStartE2EDuration="1.658081836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:48:17.655654819 +0000 UTC m=+911.309374991" watchObservedRunningTime="2026-09-10 23:48:17.658081836 +0000 UTC m=+911.311802010"
Sep 10 23:48:18.194502 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:18.194468    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:18.196216 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:18.196190    2569 patch_prober.go:28] interesting pod/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 23:48:18.196331 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:18.196237    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 23:48:18.750445 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:18.750405    2569 patch_prober.go:28] interesting pod/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 23:48:18.750668 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:18.750468    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 23:48:19.198643 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:19.198614    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:19.648756 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:19.648727    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:19.649727 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:19.649707    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:21.409906 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.409812    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:48:21.471998 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.471873    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-envoy\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.471998 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.471927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d27ee9-2167-4cc3-84af-245a86e7672c-istiod-ca-cert\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.471998 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.471971    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-socket\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.472276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472003    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-certs\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.472276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472050    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-credential-socket\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.472276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472099    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-data\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.472276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472129    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-podinfo\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.472276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472187    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-token\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.472276 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472232    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcfcv\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-kube-api-access-bcfcv\") pod \"13d27ee9-2167-4cc3-84af-245a86e7672c\" (UID: \"13d27ee9-2167-4cc3-84af-245a86e7672c\") "
Sep 10 23:48:21.472578 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472314    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:48:21.472578 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472359    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13d27ee9-2167-4cc3-84af-245a86e7672c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:48:21.472578 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472362    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:48:21.472578 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472546    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-data" (OuterVolumeSpecName: "istio-data") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:48:21.472815 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472679    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:48:21.472815 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472717    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13d27ee9-2167-4cc3-84af-245a86e7672c-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.472815 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472733    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.472815 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472745    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.472815 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.472758    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.474403 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.474374    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:48:21.474522 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.474414    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-token" (OuterVolumeSpecName: "istio-token") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:48:21.474522 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.474469    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-kube-api-access-bcfcv" (OuterVolumeSpecName: "kube-api-access-bcfcv") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "kube-api-access-bcfcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:48:21.474633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.474537    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13d27ee9-2167-4cc3-84af-245a86e7672c" (UID: "13d27ee9-2167-4cc3-84af-245a86e7672c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:48:21.573859 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.573827    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.573859 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.573851    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.573859 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.573860    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcfcv\" (UniqueName: \"kubernetes.io/projected/13d27ee9-2167-4cc3-84af-245a86e7672c-kube-api-access-bcfcv\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.573859 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.573869    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.574140 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.573877    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13d27ee9-2167-4cc3-84af-245a86e7672c-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:21.661689 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.661655    2569 generic.go:356] "Generic (PLEG): container finished" podID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerID="8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1" exitCode=0
Sep 10 23:48:21.661858 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.661698    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"
Sep 10 23:48:21.661858 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.661746    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" event={"ID":"13d27ee9-2167-4cc3-84af-245a86e7672c","Type":"ContainerDied","Data":"8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1"}
Sep 10 23:48:21.661858 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.661793    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8" event={"ID":"13d27ee9-2167-4cc3-84af-245a86e7672c","Type":"ContainerDied","Data":"d2c15d43fb4043ac71126f8272fbd9b89435eeaf323616a0bd02377c538a544c"}
Sep 10 23:48:21.661858 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.661814    2569 scope.go:117] "RemoveContainer" containerID="8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1"
Sep 10 23:48:21.671451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.671433    2569 scope.go:117] "RemoveContainer" containerID="4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1"
Sep 10 23:48:21.679654 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.679636    2569 scope.go:117] "RemoveContainer" containerID="8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1"
Sep 10 23:48:21.680058 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:48:21.680029    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1\": container with ID starting with 8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1 not found: ID does not exist" containerID="8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1"
Sep 10 23:48:21.680170 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.680066    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1"} err="failed to get container status \"8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1\": rpc error: code = NotFound desc = could not find container \"8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1\": container with ID starting with 8050ff7f18a79de81a31d8a79b413b76ff24b89f8f12e7d20dca22321ca3b2e1 not found: ID does not exist"
Sep 10 23:48:21.680170 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.680089    2569 scope.go:117] "RemoveContainer" containerID="4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1"
Sep 10 23:48:21.680568 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:48:21.680365    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1\": container with ID starting with 4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1 not found: ID does not exist" containerID="4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1"
Sep 10 23:48:21.680568 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.680391    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1"} err="failed to get container status \"4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1\": rpc error: code = NotFound desc = could not find container \"4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1\": container with ID starting with 4671831385c5c3977e507fbf12bddc8b740786bc6d254ce4b66308da287644b1 not found: ID does not exist"
Sep 10 23:48:21.682620 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.682598    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"]
Sep 10 23:48:21.686133 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:21.686113    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-abe3de-openshift-default-58cf495b46-xstf8"]
Sep 10 23:48:22.882428 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.882398    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"]
Sep 10 23:48:22.882815 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.882802    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy"
Sep 10 23:48:22.882862 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.882816    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy"
Sep 10 23:48:22.882897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.882891    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy"
Sep 10 23:48:22.887479 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.887459    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"
Sep 10 23:48:22.889996 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.889957    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:48:22.890148 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.890033    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:48:22.892455 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.892429    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"]
Sep 10 23:48:22.978435 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.978398    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" path="/var/lib/kubelet/pods/13d27ee9-2167-4cc3-84af-245a86e7672c/volumes"
Sep 10 23:48:22.988736 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:22.988699    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eefe9bb-6aab-4346-bf38-77ed9feef884-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51b15052-vlg54\" (UID: \"2eefe9bb-6aab-4346-bf38-77ed9feef884\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"
Sep 10 23:48:23.089662 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:23.089624    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eefe9bb-6aab-4346-bf38-77ed9feef884-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51b15052-vlg54\" (UID: \"2eefe9bb-6aab-4346-bf38-77ed9feef884\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"
Sep 10 23:48:23.090273 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:23.090256    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eefe9bb-6aab-4346-bf38-77ed9feef884-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-51b15052-vlg54\" (UID: \"2eefe9bb-6aab-4346-bf38-77ed9feef884\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"
Sep 10 23:48:23.199834 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:23.199748    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"
Sep 10 23:48:23.469977 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:23.469951    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"]
Sep 10 23:48:23.472119 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:48:23.472090    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eefe9bb_6aab_4346_bf38_77ed9feef884.slice/crio-80483859ed2ffc5434acc83313be2a9b0714fac1d83c77f741e4e49333d03313 WatchSource:0}: Error finding container 80483859ed2ffc5434acc83313be2a9b0714fac1d83c77f741e4e49333d03313: Status 404 returned error can't find the container with id 80483859ed2ffc5434acc83313be2a9b0714fac1d83c77f741e4e49333d03313
Sep 10 23:48:23.679281 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:23.679244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54" event={"ID":"2eefe9bb-6aab-4346-bf38-77ed9feef884","Type":"ContainerStarted","Data":"0358fbba6171b3a9935b4eb1420e2bd726172428a9b66ce04d5fef75dbcf9517"}
Sep 10 23:48:23.679444 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:23.679302    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54" event={"ID":"2eefe9bb-6aab-4346-bf38-77ed9feef884","Type":"ContainerStarted","Data":"80483859ed2ffc5434acc83313be2a9b0714fac1d83c77f741e4e49333d03313"}
Sep 10 23:48:23.693780 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:23.693727    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54" podStartSLOduration=1.693712396 podStartE2EDuration="1.693712396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:48:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:48:23.692905762 +0000 UTC m=+917.346625934" watchObservedRunningTime="2026-09-10 23:48:23.693712396 +0000 UTC m=+917.347432569"
Sep 10 23:48:24.686749 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:24.686709    2569 generic.go:356] "Generic (PLEG): container finished" podID="2eefe9bb-6aab-4346-bf38-77ed9feef884" containerID="0358fbba6171b3a9935b4eb1420e2bd726172428a9b66ce04d5fef75dbcf9517" exitCode=0
Sep 10 23:48:24.687195 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:24.686778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54" event={"ID":"2eefe9bb-6aab-4346-bf38-77ed9feef884","Type":"ContainerDied","Data":"0358fbba6171b3a9935b4eb1420e2bd726172428a9b66ce04d5fef75dbcf9517"}
Sep 10 23:48:25.832818 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:25.832792    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"
Sep 10 23:48:25.916472 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:25.916439    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eefe9bb-6aab-4346-bf38-77ed9feef884-maas-api-service-ca\") pod \"2eefe9bb-6aab-4346-bf38-77ed9feef884\" (UID: \"2eefe9bb-6aab-4346-bf38-77ed9feef884\") "
Sep 10 23:48:25.916813 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:25.916789    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2eefe9bb-6aab-4346-bf38-77ed9feef884-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2eefe9bb-6aab-4346-bf38-77ed9feef884" (UID: "2eefe9bb-6aab-4346-bf38-77ed9feef884"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:48:26.018305 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:26.018200    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2eefe9bb-6aab-4346-bf38-77ed9feef884-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:26.698457 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:26.698426    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"
Sep 10 23:48:26.698633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:26.698425    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54" event={"ID":"2eefe9bb-6aab-4346-bf38-77ed9feef884","Type":"ContainerDied","Data":"80483859ed2ffc5434acc83313be2a9b0714fac1d83c77f741e4e49333d03313"}
Sep 10 23:48:26.698633 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:26.698549    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80483859ed2ffc5434acc83313be2a9b0714fac1d83c77f741e4e49333d03313"
Sep 10 23:48:29.898628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:29.898592    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"]
Sep 10 23:48:29.899055 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:29.898840    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg" podUID="2942ec2a-8356-4417-8eea-0c2477a7bee3" containerName="payload-processing" containerID="cri-o://12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00" gracePeriod=30
Sep 10 23:48:30.052423 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.052386    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"]
Sep 10 23:48:30.056415 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.056385    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-51b15052-vlg54"]
Sep 10 23:48:30.178721 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.178693    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:30.258216 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.258186    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2942ec2a-8356-4417-8eea-0c2477a7bee3-plugins-config-volume\") pod \"2942ec2a-8356-4417-8eea-0c2477a7bee3\" (UID: \"2942ec2a-8356-4417-8eea-0c2477a7bee3\") "
Sep 10 23:48:30.258398 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.258243    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c82r\" (UniqueName: \"kubernetes.io/projected/2942ec2a-8356-4417-8eea-0c2477a7bee3-kube-api-access-4c82r\") pod \"2942ec2a-8356-4417-8eea-0c2477a7bee3\" (UID: \"2942ec2a-8356-4417-8eea-0c2477a7bee3\") "
Sep 10 23:48:30.258554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.258528    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2942ec2a-8356-4417-8eea-0c2477a7bee3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2942ec2a-8356-4417-8eea-0c2477a7bee3" (UID: "2942ec2a-8356-4417-8eea-0c2477a7bee3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:48:30.260380 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.260356    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2942ec2a-8356-4417-8eea-0c2477a7bee3-kube-api-access-4c82r" (OuterVolumeSpecName: "kube-api-access-4c82r") pod "2942ec2a-8356-4417-8eea-0c2477a7bee3" (UID: "2942ec2a-8356-4417-8eea-0c2477a7bee3"). InnerVolumeSpecName "kube-api-access-4c82r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:48:30.359640 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.359608    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c82r\" (UniqueName: \"kubernetes.io/projected/2942ec2a-8356-4417-8eea-0c2477a7bee3-kube-api-access-4c82r\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:30.359640 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.359635    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2942ec2a-8356-4417-8eea-0c2477a7bee3-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:48:30.720163 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.720129    2569 generic.go:356] "Generic (PLEG): container finished" podID="2942ec2a-8356-4417-8eea-0c2477a7bee3" containerID="12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00" exitCode=0
Sep 10 23:48:30.720352 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.720173    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"
Sep 10 23:48:30.720352 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.720216    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg" event={"ID":"2942ec2a-8356-4417-8eea-0c2477a7bee3","Type":"ContainerDied","Data":"12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00"}
Sep 10 23:48:30.720352 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.720268    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg" event={"ID":"2942ec2a-8356-4417-8eea-0c2477a7bee3","Type":"ContainerDied","Data":"13dadba4e607653fbe0b6e7dbbdfbf518fa8ac2dd71d33bb8f1469d811221717"}
Sep 10 23:48:30.720352 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.720290    2569 scope.go:117] "RemoveContainer" containerID="12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00"
Sep 10 23:48:30.732694 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.732670    2569 scope.go:117] "RemoveContainer" containerID="12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00"
Sep 10 23:48:30.733049 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:48:30.733025    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00\": container with ID starting with 12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00 not found: ID does not exist" containerID="12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00"
Sep 10 23:48:30.733143 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.733058    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00"} err="failed to get container status \"12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00\": rpc error: code = NotFound desc = could not find container \"12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00\": container with ID starting with 12c176f26e16cf1f05e85b5888e4d42297a351e1f97e786bc720aaef5df56c00 not found: ID does not exist"
Sep 10 23:48:30.741630 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.741600    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"]
Sep 10 23:48:30.744493 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.744467    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-51b15052-5d6959d84f-sj2pg"]
Sep 10 23:48:30.978658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.978577    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2942ec2a-8356-4417-8eea-0c2477a7bee3" path="/var/lib/kubelet/pods/2942ec2a-8356-4417-8eea-0c2477a7bee3/volumes"
Sep 10 23:48:30.979061 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:30.978914    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2eefe9bb-6aab-4346-bf38-77ed9feef884" path="/var/lib/kubelet/pods/2eefe9bb-6aab-4346-bf38-77ed9feef884/volumes"
Sep 10 23:48:44.807018 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:44.806904    2569 generic.go:356] "Generic (PLEG): container finished" podID="f77f243a-285b-4ab8-a85e-276079d16d79" containerID="cb55b8bac0a46cafdafe6dc8fdffca8fdde9b9002657dd954a5c1ff0e916ca0f" exitCode=0
Sep 10 23:48:44.807018 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:44.806983    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" event={"ID":"f77f243a-285b-4ab8-a85e-276079d16d79","Type":"ContainerDied","Data":"cb55b8bac0a46cafdafe6dc8fdffca8fdde9b9002657dd954a5c1ff0e916ca0f"}
Sep 10 23:48:44.807444 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:44.807394    2569 scope.go:117] "RemoveContainer" containerID="cb55b8bac0a46cafdafe6dc8fdffca8fdde9b9002657dd954a5c1ff0e916ca0f"
Sep 10 23:48:44.808498 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:44.808468    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:48:44.808616 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:44.808562    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:48:44.808685 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:44.808624    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:48:45.813589 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:45.813547    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" event={"ID":"f77f243a-285b-4ab8-a85e-276079d16d79","Type":"ContainerStarted","Data":"223ef72b4c40825df1240e1c309f0b959efa70396ed57937d511effa6dc3cf59"}
Sep 10 23:48:46.193629 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:46.193592    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:46.194634 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:46.194613    2569 patch_prober.go:28] interesting pod/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 23:48:46.194702 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:46.194660    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 23:48:47.194041 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:47.194005    2569 patch_prober.go:28] interesting pod/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 23:48:47.194435 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:47.194064    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 23:48:48.194418 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:48.194380    2569 patch_prober.go:28] interesting pod/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 23:48:48.194812 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:48.194450    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 23:48:49.194840 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:49.194803    2569 patch_prober.go:28] interesting pod/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 23:48:49.195299 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:49.194861    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 23:48:49.649319 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:49.649270    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:50.194020 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:50.193979    2569 patch_prober.go:28] interesting pod/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 10 23:48:50.194235 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:50.194055    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 10 23:48:51.198617 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:51.198588    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:48:51.199803 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:48:51.199780    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:49:11.415832 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.415789    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"]
Sep 10 23:49:11.416395 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416372    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy"
Sep 10 23:49:11.416476 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416399    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy"
Sep 10 23:49:11.416476 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416428    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2942ec2a-8356-4417-8eea-0c2477a7bee3" containerName="payload-processing"
Sep 10 23:49:11.416476 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416440    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2942ec2a-8356-4417-8eea-0c2477a7bee3" containerName="payload-processing"
Sep 10 23:49:11.416476 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416466    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2eefe9bb-6aab-4346-bf38-77ed9feef884" containerName="revoke-keys"
Sep 10 23:49:11.416476 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416475    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2eefe9bb-6aab-4346-bf38-77ed9feef884" containerName="revoke-keys"
Sep 10 23:49:11.416729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416576    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="13d27ee9-2167-4cc3-84af-245a86e7672c" containerName="istio-proxy"
Sep 10 23:49:11.416729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416590    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2942ec2a-8356-4417-8eea-0c2477a7bee3" containerName="payload-processing"
Sep 10 23:49:11.416729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.416600    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2eefe9bb-6aab-4346-bf38-77ed9feef884" containerName="revoke-keys"
Sep 10 23:49:11.421260 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.421238    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"
Sep 10 23:49:11.423225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.423203    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:49:11.423225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.423214    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:49:11.427865 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.427844    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"]
Sep 10 23:49:11.457498 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.457458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98bc3d-7550-4adf-82af-da8d823e0741-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b\" (UID: \"ff98bc3d-7550-4adf-82af-da8d823e0741\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"
Sep 10 23:49:11.559418 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.559379    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98bc3d-7550-4adf-82af-da8d823e0741-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b\" (UID: \"ff98bc3d-7550-4adf-82af-da8d823e0741\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"
Sep 10 23:49:11.560066 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.560043    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98bc3d-7550-4adf-82af-da8d823e0741-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b\" (UID: \"ff98bc3d-7550-4adf-82af-da8d823e0741\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"
Sep 10 23:49:11.732626 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:11.732530    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"
Sep 10 23:49:12.001774 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:12.001750    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"]
Sep 10 23:49:12.004028 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:49:12.003993    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff98bc3d_7550_4adf_82af_da8d823e0741.slice/crio-0f00e2f5d92656e9e7f2a66bf638f0790bcc70d20bcb9d10c9ad0dfcff715c7c WatchSource:0}: Error finding container 0f00e2f5d92656e9e7f2a66bf638f0790bcc70d20bcb9d10c9ad0dfcff715c7c: Status 404 returned error can't find the container with id 0f00e2f5d92656e9e7f2a66bf638f0790bcc70d20bcb9d10c9ad0dfcff715c7c
Sep 10 23:49:12.962764 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:12.962724    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff98bc3d-7550-4adf-82af-da8d823e0741" containerID="d68e0ef55801dfcf1f351caab6ab9f8284240a84ab96e3f2f9cf4b6ad02f52f7" exitCode=0
Sep 10 23:49:12.963179 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:12.962813    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b" event={"ID":"ff98bc3d-7550-4adf-82af-da8d823e0741","Type":"ContainerDied","Data":"d68e0ef55801dfcf1f351caab6ab9f8284240a84ab96e3f2f9cf4b6ad02f52f7"}
Sep 10 23:49:12.963179 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:12.962895    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b" event={"ID":"ff98bc3d-7550-4adf-82af-da8d823e0741","Type":"ContainerStarted","Data":"0f00e2f5d92656e9e7f2a66bf638f0790bcc70d20bcb9d10c9ad0dfcff715c7c"}
Sep 10 23:49:14.107232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:14.107207    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"
Sep 10 23:49:14.184411 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:14.184379    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98bc3d-7550-4adf-82af-da8d823e0741-maas-api-service-ca\") pod \"ff98bc3d-7550-4adf-82af-da8d823e0741\" (UID: \"ff98bc3d-7550-4adf-82af-da8d823e0741\") "
Sep 10 23:49:14.184746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:14.184724    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff98bc3d-7550-4adf-82af-da8d823e0741-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff98bc3d-7550-4adf-82af-da8d823e0741" (UID: "ff98bc3d-7550-4adf-82af-da8d823e0741"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:49:14.286347 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:14.286256    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff98bc3d-7550-4adf-82af-da8d823e0741-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:14.974627 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:14.974600    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"
Sep 10 23:49:14.978297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:14.978267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b" event={"ID":"ff98bc3d-7550-4adf-82af-da8d823e0741","Type":"ContainerDied","Data":"0f00e2f5d92656e9e7f2a66bf638f0790bcc70d20bcb9d10c9ad0dfcff715c7c"}
Sep 10 23:49:14.978441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:14.978311    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f00e2f5d92656e9e7f2a66bf638f0790bcc70d20bcb9d10c9ad0dfcff715c7c"
Sep 10 23:49:22.279502 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:22.279466    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"]
Sep 10 23:49:22.282565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:22.282542    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-a66ef6-6fq4b"]
Sep 10 23:49:22.978142 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:22.978109    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff98bc3d-7550-4adf-82af-da8d823e0741" path="/var/lib/kubelet/pods/ff98bc3d-7550-4adf-82af-da8d823e0741/volumes"
Sep 10 23:49:28.937818 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:28.937781    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"]
Sep 10 23:49:28.938330 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:28.938020    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy" containerID="cri-o://223ef72b4c40825df1240e1c309f0b959efa70396ed57937d511effa6dc3cf59" gracePeriod=30
Sep 10 23:49:29.624949 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.624907    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"]
Sep 10 23:49:29.625537 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.625521    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff98bc3d-7550-4adf-82af-da8d823e0741" containerName="revoke-keys"
Sep 10 23:49:29.625611 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.625541    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff98bc3d-7550-4adf-82af-da8d823e0741" containerName="revoke-keys"
Sep 10 23:49:29.625671 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.625626    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff98bc3d-7550-4adf-82af-da8d823e0741" containerName="revoke-keys"
Sep 10 23:49:29.630177 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.630150    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.634703 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.634676    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-a70fc0-openshift-default-dockercfg-s2q58\""
Sep 10 23:49:29.658743 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.658708    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"]
Sep 10 23:49:29.740366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.740309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istiod-ca-cert\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.740659 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.740640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-podinfo\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.740781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.740768    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-credential-socket\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.740891 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.740876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-envoy\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.741021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.741007    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-data\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.741186 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.741165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-token\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.741328 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.741200    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-socket\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.741328 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.741271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-certs\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.741328 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.741312    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc5kd\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-kube-api-access-nc5kd\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842480 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-data\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842676 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-token\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842676 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842557    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-socket\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842676 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842623    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-certs\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842676 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc5kd\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-kube-api-access-nc5kd\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842884 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842705    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istiod-ca-cert\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842884 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842743    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-podinfo\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842884 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-credential-socket\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.842884 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842807    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-envoy\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.843115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.842904    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-data\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.843115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.843008    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-certs\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.843115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.843092    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-socket\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.843268 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.843248    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-credential-socket\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.843492 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.843464    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istiod-ca-cert\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.845264 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.845235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-envoy\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.845569 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.845547    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-podinfo\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.851201 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.851177    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-token\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.851429 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.851407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc5kd\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-kube-api-access-nc5kd\") pod \"e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") " pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:29.941685 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:29.941600    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:30.219705 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:30.219680    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"]
Sep 10 23:49:30.221583 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:49:30.221555    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a66e25e_5ba8_41a4_b606_4a3842979f8d.slice/crio-3454d2ee963815d5a01206c07735c45404227d07bb456916c7156dc150d5a69f WatchSource:0}: Error finding container 3454d2ee963815d5a01206c07735c45404227d07bb456916c7156dc150d5a69f: Status 404 returned error can't find the container with id 3454d2ee963815d5a01206c07735c45404227d07bb456916c7156dc150d5a69f
Sep 10 23:49:30.223852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:30.223817    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:49:30.223924 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:30.223905    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:49:30.224001 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:30.223975    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:49:31.063441 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:31.063401    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9" event={"ID":"8a66e25e-5ba8-41a4-b606-4a3842979f8d","Type":"ContainerStarted","Data":"d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0"}
Sep 10 23:49:31.063808 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:31.063464    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9" event={"ID":"8a66e25e-5ba8-41a4-b606-4a3842979f8d","Type":"ContainerStarted","Data":"3454d2ee963815d5a01206c07735c45404227d07bb456916c7156dc150d5a69f"}
Sep 10 23:49:31.082127 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:31.082075    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9" podStartSLOduration=2.082060131 podStartE2EDuration="2.082060131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:49:31.08010459 +0000 UTC m=+984.733824767" watchObservedRunningTime="2026-09-10 23:49:31.082060131 +0000 UTC m=+984.735780303"
Sep 10 23:49:31.942701 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:31.942666    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:31.948680 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:31.948651    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:32.069315 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:32.069284    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:32.070464 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:32.070443    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:34.081295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.081255    2569 generic.go:356] "Generic (PLEG): container finished" podID="f77f243a-285b-4ab8-a85e-276079d16d79" containerID="223ef72b4c40825df1240e1c309f0b959efa70396ed57937d511effa6dc3cf59" exitCode=0
Sep 10 23:49:34.081766 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.081489    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" event={"ID":"f77f243a-285b-4ab8-a85e-276079d16d79","Type":"ContainerDied","Data":"223ef72b4c40825df1240e1c309f0b959efa70396ed57937d511effa6dc3cf59"}
Sep 10 23:49:34.081766 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.081556    2569 scope.go:117] "RemoveContainer" containerID="cb55b8bac0a46cafdafe6dc8fdffca8fdde9b9002657dd954a5c1ff0e916ca0f"
Sep 10 23:49:34.197392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.197368    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:49:34.283082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.282993    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkdnw\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-kube-api-access-gkdnw\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283043    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f77f243a-285b-4ab8-a85e-276079d16d79-istio-podinfo\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283074    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-socket\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283358 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283102    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-certs\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283358 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283126    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-credential-socket\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283358 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283143    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-data\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283358 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283183    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-istio-token\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283358 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283215    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-envoy\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283358 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283314    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f77f243a-285b-4ab8-a85e-276079d16d79-istiod-ca-cert\") pod \"f77f243a-285b-4ab8-a85e-276079d16d79\" (UID: \"f77f243a-285b-4ab8-a85e-276079d16d79\") "
Sep 10 23:49:34.283650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283452    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:34.283650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283495    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:34.283650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283499    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:34.283650 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283511    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-data" (OuterVolumeSpecName: "istio-data") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:34.283888 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283740    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:34.283888 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283762    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:34.283888 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283762    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f77f243a-285b-4ab8-a85e-276079d16d79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:49:34.283888 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283776    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:34.283888 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.283788    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:34.285478 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.285454    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f77f243a-285b-4ab8-a85e-276079d16d79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:49:34.285582 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.285531    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-kube-api-access-gkdnw" (OuterVolumeSpecName: "kube-api-access-gkdnw") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "kube-api-access-gkdnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:49:34.285832 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.285812    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:34.285832 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.285820    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-istio-token" (OuterVolumeSpecName: "istio-token") pod "f77f243a-285b-4ab8-a85e-276079d16d79" (UID: "f77f243a-285b-4ab8-a85e-276079d16d79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:49:34.384709 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.384676    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f77f243a-285b-4ab8-a85e-276079d16d79-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:34.384709 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.384703    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkdnw\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-kube-api-access-gkdnw\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:34.384709 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.384714    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f77f243a-285b-4ab8-a85e-276079d16d79-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:34.384960 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.384722    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f77f243a-285b-4ab8-a85e-276079d16d79-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:34.384960 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:34.384731    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f77f243a-285b-4ab8-a85e-276079d16d79-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:35.087774 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:35.087742    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"
Sep 10 23:49:35.088297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:35.087774    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8" event={"ID":"f77f243a-285b-4ab8-a85e-276079d16d79","Type":"ContainerDied","Data":"b2bd3f8dc714e287f1a8a99e0fa03562b076c0749bd7c7d7eae3e57c2448afad"}
Sep 10 23:49:35.088297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:35.087846    2569 scope.go:117] "RemoveContainer" containerID="223ef72b4c40825df1240e1c309f0b959efa70396ed57937d511effa6dc3cf59"
Sep 10 23:49:35.103863 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:35.103840    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"]
Sep 10 23:49:35.110059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:35.110038    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-a66ef6-openshift-default-65746d48cb-kmsl8"]
Sep 10 23:49:36.980439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:36.980403    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" path="/var/lib/kubelet/pods/f77f243a-285b-4ab8-a85e-276079d16d79/volumes"
Sep 10 23:49:47.683195 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:47.683157    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"]
Sep 10 23:49:47.683725 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:47.683477    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9" podUID="8a66e25e-5ba8-41a4-b606-4a3842979f8d" containerName="istio-proxy" containerID="cri-o://d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0" gracePeriod=30
Sep 10 23:49:52.949343 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:52.949281    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:53.067873 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.067833    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-podinfo\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.067890    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-data\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.067961    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-envoy\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.067991    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-socket\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068028    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-token\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068046    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc5kd\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-kube-api-access-nc5kd\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068075    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-certs\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068401 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068146    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-credential-socket\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068401 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068186    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istiod-ca-cert\") pod \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\" (UID: \"8a66e25e-5ba8-41a4-b606-4a3842979f8d\") "
Sep 10 23:49:53.068401 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068325    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:53.068401 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068389    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:53.068638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068594    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.068638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068601    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:53.068638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068616    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.068801 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068642    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:49:53.068801 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.068674    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-data" (OuterVolumeSpecName: "istio-data") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:53.070389 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.070366    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-token" (OuterVolumeSpecName: "istio-token") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:49:53.070523 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.070402    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-kube-api-access-nc5kd" (OuterVolumeSpecName: "kube-api-access-nc5kd") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "kube-api-access-nc5kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:49:53.070587 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.070521    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:49:53.070587 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.070532    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8a66e25e-5ba8-41a4-b606-4a3842979f8d" (UID: "8a66e25e-5ba8-41a4-b606-4a3842979f8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:49:53.169915 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.169876    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.169915 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.169906    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.169915 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.169914    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.169915 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.169922    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.170211 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.169930    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc5kd\" (UniqueName: \"kubernetes.io/projected/8a66e25e-5ba8-41a4-b606-4a3842979f8d-kube-api-access-nc5kd\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.170211 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.169981    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a66e25e-5ba8-41a4-b606-4a3842979f8d-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.170211 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.169995    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a66e25e-5ba8-41a4-b606-4a3842979f8d-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:49:53.192690 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.192656    2569 generic.go:356] "Generic (PLEG): container finished" podID="8a66e25e-5ba8-41a4-b606-4a3842979f8d" containerID="d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0" exitCode=0
Sep 10 23:49:53.192852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.192708    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"
Sep 10 23:49:53.192852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.192708    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9" event={"ID":"8a66e25e-5ba8-41a4-b606-4a3842979f8d","Type":"ContainerDied","Data":"d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0"}
Sep 10 23:49:53.192852 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.192835    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9" event={"ID":"8a66e25e-5ba8-41a4-b606-4a3842979f8d","Type":"ContainerDied","Data":"3454d2ee963815d5a01206c07735c45404227d07bb456916c7156dc150d5a69f"}
Sep 10 23:49:53.193001 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.192865    2569 scope.go:117] "RemoveContainer" containerID="d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0"
Sep 10 23:49:53.202447 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.202429    2569 scope.go:117] "RemoveContainer" containerID="d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0"
Sep 10 23:49:53.202747 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:49:53.202726    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0\": container with ID starting with d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0 not found: ID does not exist" containerID="d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0"
Sep 10 23:49:53.202786 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.202758    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0"} err="failed to get container status \"d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0\": rpc error: code = NotFound desc = could not find container \"d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0\": container with ID starting with d78104ee9af19383b043f6715f868ffb364129eb858c46ec1b709ccbfa03aae0 not found: ID does not exist"
Sep 10 23:49:53.213284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.213255    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"]
Sep 10 23:49:53.215149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:53.215126    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-a70fc0-openshift-default-8c49cdd9d-8hgb9"]
Sep 10 23:49:54.980741 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:49:54.980702    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a66e25e-5ba8-41a4-b606-4a3842979f8d" path="/var/lib/kubelet/pods/8a66e25e-5ba8-41a4-b606-4a3842979f8d/volumes"
Sep 10 23:50:05.511669 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.511634    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"]
Sep 10 23:50:05.512056 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.512047    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy"
Sep 10 23:50:05.512106 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.512059    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy"
Sep 10 23:50:05.512106 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.512077    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a66e25e-5ba8-41a4-b606-4a3842979f8d" containerName="istio-proxy"
Sep 10 23:50:05.512106 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.512083    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a66e25e-5ba8-41a4-b606-4a3842979f8d" containerName="istio-proxy"
Sep 10 23:50:05.512202 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.512143    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a66e25e-5ba8-41a4-b606-4a3842979f8d" containerName="istio-proxy"
Sep 10 23:50:05.512202 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.512155    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy"
Sep 10 23:50:05.512202 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.512161    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy"
Sep 10 23:50:05.515147 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.515133    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"
Sep 10 23:50:05.517470 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.517445    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:50:05.517598 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.517518    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:50:05.523033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.523011    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"]
Sep 10 23:50:05.594463 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.594427    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe9567a-ad62-47ee-967d-9a0257282b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-487960-zlk5r\" (UID: \"abe9567a-ad62-47ee-967d-9a0257282b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"
Sep 10 23:50:05.695101 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.695068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe9567a-ad62-47ee-967d-9a0257282b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-487960-zlk5r\" (UID: \"abe9567a-ad62-47ee-967d-9a0257282b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"
Sep 10 23:50:05.695748 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.695721    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe9567a-ad62-47ee-967d-9a0257282b0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-487960-zlk5r\" (UID: \"abe9567a-ad62-47ee-967d-9a0257282b0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"
Sep 10 23:50:05.827023 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.826907    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"
Sep 10 23:50:05.889609 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.889580    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"]
Sep 10 23:50:05.890074 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.890056    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy"
Sep 10 23:50:05.890176 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.890076    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f77f243a-285b-4ab8-a85e-276079d16d79" containerName="istio-proxy"
Sep 10 23:50:05.895300 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.895279    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.897858 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.897667    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e5eaec01-openshift-default-dockercfg-hwdkq\""
Sep 10 23:50:05.905824 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.905803    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"]
Sep 10 23:50:05.998544 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.998507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istiod-ca-cert\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.998827 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.998797    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-envoy\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.999059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.998836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-token\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.999059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.998863    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-certs\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.999059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.998895    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kn7b\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-kube-api-access-6kn7b\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.999059 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.998946    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-credential-socket\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.999306 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.999061    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-data\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.999306 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.999103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-socket\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:05.999306 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:05.999186    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-podinfo\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100388 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100308    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-podinfo\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100388 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100354    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istiod-ca-cert\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100388 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100390    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-envoy\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100669 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100406    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-token\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100669 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100424    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-certs\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100669 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100445    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kn7b\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-kube-api-access-6kn7b\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100669 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100477    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-credential-socket\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100974 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-data\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100974 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100752    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-socket\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.100974 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.100971    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-credential-socket\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.101148 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.101009    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-certs\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.101232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.101210    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-socket\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.101304 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.101229    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-data\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.101425 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.101404    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istiod-ca-cert\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.103328 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.103300    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-envoy\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.103790 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.103770    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-podinfo\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.105041 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.105022    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"]
Sep 10 23:50:06.107606 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:50:06.107125    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe9567a_ad62_47ee_967d_9a0257282b0c.slice/crio-e9c0a80fa6ba2ad600425b84dd63b42ec6c2cf292f29e810d50131d83089573e WatchSource:0}: Error finding container e9c0a80fa6ba2ad600425b84dd63b42ec6c2cf292f29e810d50131d83089573e: Status 404 returned error can't find the container with id e9c0a80fa6ba2ad600425b84dd63b42ec6c2cf292f29e810d50131d83089573e
Sep 10 23:50:06.114451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.109978    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-token\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.114451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.110237    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kn7b\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-kube-api-access-6kn7b\") pod \"e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") " pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.213355 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.213316    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:06.259844 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.259807    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r" event={"ID":"abe9567a-ad62-47ee-967d-9a0257282b0c","Type":"ContainerStarted","Data":"e9c0a80fa6ba2ad600425b84dd63b42ec6c2cf292f29e810d50131d83089573e"}
Sep 10 23:50:06.487669 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.487646    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"]
Sep 10 23:50:06.489738 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:50:06.489709    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c45e003_91c4_486c_89a7_3ff56e2a0bbc.slice/crio-44ddb76d57ea20c163e7405e96173ed4e53fa6bda43134ae8bc3f4bff37b5f5d WatchSource:0}: Error finding container 44ddb76d57ea20c163e7405e96173ed4e53fa6bda43134ae8bc3f4bff37b5f5d: Status 404 returned error can't find the container with id 44ddb76d57ea20c163e7405e96173ed4e53fa6bda43134ae8bc3f4bff37b5f5d
Sep 10 23:50:06.492011 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.491982    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:06.492119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.492053    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:06.492119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:06.492094    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:07.266439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:07.266403    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" event={"ID":"4c45e003-91c4-486c-89a7-3ff56e2a0bbc","Type":"ContainerStarted","Data":"9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f"}
Sep 10 23:50:07.266889 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:07.266460    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" event={"ID":"4c45e003-91c4-486c-89a7-3ff56e2a0bbc","Type":"ContainerStarted","Data":"44ddb76d57ea20c163e7405e96173ed4e53fa6bda43134ae8bc3f4bff37b5f5d"}
Sep 10 23:50:07.267882 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:07.267857    2569 generic.go:356] "Generic (PLEG): container finished" podID="abe9567a-ad62-47ee-967d-9a0257282b0c" containerID="23d4ca2d688a90f8af2916c1d7472727b195810bb3cef1fb5c49fea297046607" exitCode=0
Sep 10 23:50:07.268010 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:07.267901    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r" event={"ID":"abe9567a-ad62-47ee-967d-9a0257282b0c","Type":"ContainerDied","Data":"23d4ca2d688a90f8af2916c1d7472727b195810bb3cef1fb5c49fea297046607"}
Sep 10 23:50:07.285565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:07.285519    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podStartSLOduration=2.285502671 podStartE2EDuration="2.285502671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:07.284757449 +0000 UTC m=+1020.938477619" watchObservedRunningTime="2026-09-10 23:50:07.285502671 +0000 UTC m=+1020.939222845"
Sep 10 23:50:08.213948 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:08.213908    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:08.219028 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:08.219004    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:08.273137 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:08.273106    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:08.274011 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:08.273991    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:08.415697 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:08.415672    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"
Sep 10 23:50:08.524204 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:08.524108    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe9567a-ad62-47ee-967d-9a0257282b0c-maas-api-service-ca\") pod \"abe9567a-ad62-47ee-967d-9a0257282b0c\" (UID: \"abe9567a-ad62-47ee-967d-9a0257282b0c\") "
Sep 10 23:50:08.524490 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:08.524466    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abe9567a-ad62-47ee-967d-9a0257282b0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "abe9567a-ad62-47ee-967d-9a0257282b0c" (UID: "abe9567a-ad62-47ee-967d-9a0257282b0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:50:08.625862 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:08.625825    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/abe9567a-ad62-47ee-967d-9a0257282b0c-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:09.279365 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:09.279268    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"
Sep 10 23:50:09.279365 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:09.279308    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r" event={"ID":"abe9567a-ad62-47ee-967d-9a0257282b0c","Type":"ContainerDied","Data":"e9c0a80fa6ba2ad600425b84dd63b42ec6c2cf292f29e810d50131d83089573e"}
Sep 10 23:50:09.279365 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:09.279365    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9c0a80fa6ba2ad600425b84dd63b42ec6c2cf292f29e810d50131d83089573e"
Sep 10 23:50:10.962101 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:10.962072    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"]
Sep 10 23:50:10.962487 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:10.962472    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe9567a-ad62-47ee-967d-9a0257282b0c" containerName="revoke-keys"
Sep 10 23:50:10.962531 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:10.962489    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe9567a-ad62-47ee-967d-9a0257282b0c" containerName="revoke-keys"
Sep 10 23:50:10.962576 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:10.962568    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="abe9567a-ad62-47ee-967d-9a0257282b0c" containerName="revoke-keys"
Sep 10 23:50:10.965712 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:10.965697    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:10.968126 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:10.968101    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e5eaec01-dockercfg-kj47h\""
Sep 10 23:50:10.968261 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:10.968211    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e5eaec01\""
Sep 10 23:50:10.979853 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:10.979827    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"]
Sep 10 23:50:11.149177 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:11.149144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43687a4c-639f-45cd-8179-6998d7bc0afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff\" (UID: \"43687a4c-639f-45cd-8179-6998d7bc0afc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:11.149366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:11.149347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2mgz\" (UniqueName: \"kubernetes.io/projected/43687a4c-639f-45cd-8179-6998d7bc0afc-kube-api-access-k2mgz\") pod \"payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff\" (UID: \"43687a4c-639f-45cd-8179-6998d7bc0afc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:11.250184 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:11.250084    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43687a4c-639f-45cd-8179-6998d7bc0afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff\" (UID: \"43687a4c-639f-45cd-8179-6998d7bc0afc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:11.250349 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:11.250257    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2mgz\" (UniqueName: \"kubernetes.io/projected/43687a4c-639f-45cd-8179-6998d7bc0afc-kube-api-access-k2mgz\") pod \"payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff\" (UID: \"43687a4c-639f-45cd-8179-6998d7bc0afc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:11.250771 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:11.250751    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43687a4c-639f-45cd-8179-6998d7bc0afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff\" (UID: \"43687a4c-639f-45cd-8179-6998d7bc0afc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:11.257807 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:11.257783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2mgz\" (UniqueName: \"kubernetes.io/projected/43687a4c-639f-45cd-8179-6998d7bc0afc-kube-api-access-k2mgz\") pod \"payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff\" (UID: \"43687a4c-639f-45cd-8179-6998d7bc0afc\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:11.276081 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:11.276052    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:11.547637 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:11.547611    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"]
Sep 10 23:50:11.550350 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:50:11.550323    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43687a4c_639f_45cd_8179_6998d7bc0afc.slice/crio-d6aa76c4c50502cac06bf634ca44eb559d8e6d53a96633b77dbee5044338387b WatchSource:0}: Error finding container d6aa76c4c50502cac06bf634ca44eb559d8e6d53a96633b77dbee5044338387b: Status 404 returned error can't find the container with id d6aa76c4c50502cac06bf634ca44eb559d8e6d53a96633b77dbee5044338387b
Sep 10 23:50:12.298295 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:12.298259    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff" event={"ID":"43687a4c-639f-45cd-8179-6998d7bc0afc","Type":"ContainerStarted","Data":"b70c644e4b52a24df39bd47c2217b4f97a302b10d44b2be0d3e00db843ae37ec"}
Sep 10 23:50:12.298710 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:12.298309    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff" event={"ID":"43687a4c-639f-45cd-8179-6998d7bc0afc","Type":"ContainerStarted","Data":"d6aa76c4c50502cac06bf634ca44eb559d8e6d53a96633b77dbee5044338387b"}
Sep 10 23:50:12.298710 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:12.298350    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:12.313364 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:12.313303    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff" podStartSLOduration=2.313257267 podStartE2EDuration="2.313257267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:12.312732665 +0000 UTC m=+1025.966452838" watchObservedRunningTime="2026-09-10 23:50:12.313257267 +0000 UTC m=+1025.966977440"
Sep 10 23:50:16.478158 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.478126    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"]
Sep 10 23:50:16.478718 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.478376    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj" podUID="c8ed38e3-dfb1-4827-872e-5ecaa74e9c29" containerName="payload-processing" containerID="cri-o://0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b" gracePeriod=30
Sep 10 23:50:16.628323 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.628286    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"]
Sep 10 23:50:16.632435 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.632408    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-487960-zlk5r"]
Sep 10 23:50:16.757177 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.757143    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:50:16.809751 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.809721    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8s6v\" (UniqueName: \"kubernetes.io/projected/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-kube-api-access-k8s6v\") pod \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\" (UID: \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\") "
Sep 10 23:50:16.809751 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.809755    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-plugins-config-volume\") pod \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\" (UID: \"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29\") "
Sep 10 23:50:16.810173 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.810154    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c8ed38e3-dfb1-4827-872e-5ecaa74e9c29" (UID: "c8ed38e3-dfb1-4827-872e-5ecaa74e9c29"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:50:16.811860 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.811840    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-kube-api-access-k8s6v" (OuterVolumeSpecName: "kube-api-access-k8s6v") pod "c8ed38e3-dfb1-4827-872e-5ecaa74e9c29" (UID: "c8ed38e3-dfb1-4827-872e-5ecaa74e9c29"). InnerVolumeSpecName "kube-api-access-k8s6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:50:16.910667 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.910630    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8s6v\" (UniqueName: \"kubernetes.io/projected/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-kube-api-access-k8s6v\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:16.910667 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.910657    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:16.978979 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:16.978922    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abe9567a-ad62-47ee-967d-9a0257282b0c" path="/var/lib/kubelet/pods/abe9567a-ad62-47ee-967d-9a0257282b0c/volumes"
Sep 10 23:50:17.326139 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.326049    2569 generic.go:356] "Generic (PLEG): container finished" podID="c8ed38e3-dfb1-4827-872e-5ecaa74e9c29" containerID="0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b" exitCode=0
Sep 10 23:50:17.326139 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.326102    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"
Sep 10 23:50:17.326319 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.326130    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj" event={"ID":"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29","Type":"ContainerDied","Data":"0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b"}
Sep 10 23:50:17.326319 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.326185    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj" event={"ID":"c8ed38e3-dfb1-4827-872e-5ecaa74e9c29","Type":"ContainerDied","Data":"5043cf9f71f9509ee91bfbba60b72af964ec62810a2ae4e479264c642520836a"}
Sep 10 23:50:17.326319 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.326207    2569 scope.go:117] "RemoveContainer" containerID="0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b"
Sep 10 23:50:17.336634 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.336450    2569 scope.go:117] "RemoveContainer" containerID="0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b"
Sep 10 23:50:17.342046 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:50:17.337111    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b\": container with ID starting with 0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b not found: ID does not exist" containerID="0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b"
Sep 10 23:50:17.342046 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.337147    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b"} err="failed to get container status \"0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b\": rpc error: code = NotFound desc = could not find container \"0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b\": container with ID starting with 0087a14b43bd0d7debe40914c26941141bf11f42ca73a760c6b866fd36cafd1b not found: ID does not exist"
Sep 10 23:50:17.353046 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.352987    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"]
Sep 10 23:50:17.353046 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:17.353050    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-487960-59f69564f7-z7ssj"]
Sep 10 23:50:18.978791 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:18.978756    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ed38e3-dfb1-4827-872e-5ecaa74e9c29" path="/var/lib/kubelet/pods/c8ed38e3-dfb1-4827-872e-5ecaa74e9c29/volumes"
Sep 10 23:50:23.119000 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:23.118928    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"]
Sep 10 23:50:23.119458 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:23.119240    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" containerID="cri-o://a812cb8ae666cd8c7ba86f006c90252038d2e23cb67faee05252c934a7607ab1" gracePeriod=30
Sep 10 23:50:23.306883 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:23.306850    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:24.310233 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:24.310191    2569 patch_prober.go:28] interesting pod/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 23:50:24.310606 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:24.310267    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 23:50:24.314060 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:24.314035    2569 patch_prober.go:28] interesting pod/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 23:50:24.314137 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:24.314081    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 23:50:26.385840 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:26.385801    2569 generic.go:356] "Generic (PLEG): container finished" podID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerID="9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f" exitCode=0
Sep 10 23:50:26.386279 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:26.385871    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" event={"ID":"4c45e003-91c4-486c-89a7-3ff56e2a0bbc","Type":"ContainerDied","Data":"9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f"}
Sep 10 23:50:26.386378 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:26.386360    2569 scope.go:117] "RemoveContainer" containerID="9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f"
Sep 10 23:50:26.387423 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:26.387398    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:26.387485 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:26.387463    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:26.387524 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:26.387492    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:27.392452 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:27.392415    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" event={"ID":"4c45e003-91c4-486c-89a7-3ff56e2a0bbc","Type":"ContainerStarted","Data":"f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b"}
Sep 10 23:50:28.214438 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.214398    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:28.214674 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.214652    2569 patch_prober.go:28] interesting pod/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 23:50:28.214734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.214714    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 23:50:28.400224 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.400192    2569 generic.go:356] "Generic (PLEG): container finished" podID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerID="a812cb8ae666cd8c7ba86f006c90252038d2e23cb67faee05252c934a7607ab1" exitCode=0
Sep 10 23:50:28.400635 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.400264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" event={"ID":"5591bae4-a799-462a-b00d-fe0dff4cdbb3","Type":"ContainerDied","Data":"a812cb8ae666cd8c7ba86f006c90252038d2e23cb67faee05252c934a7607ab1"}
Sep 10 23:50:28.400635 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.400342    2569 scope.go:117] "RemoveContainer" containerID="627a2ee07a45d59644d968c83fef56fd09e4c29727a6c16168e403b21b0b7ab7"
Sep 10 23:50:28.419436 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:50:28.419414    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"627a2ee07a45d59644d968c83fef56fd09e4c29727a6c16168e403b21b0b7ab7\": container with ID starting with 627a2ee07a45d59644d968c83fef56fd09e4c29727a6c16168e403b21b0b7ab7 not found: ID does not exist" containerID="627a2ee07a45d59644d968c83fef56fd09e4c29727a6c16168e403b21b0b7ab7"
Sep 10 23:50:28.421255 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.421235    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:50:28.531339 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531298    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-credential-socket\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.531339 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531351    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-envoy\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.531605 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531390    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-certs\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.531605 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531411    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istiod-ca-cert\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.531605 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531503    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-token\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.531605 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531522    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxk85\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-kube-api-access-dxk85\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.531605 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531556    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-podinfo\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.531849 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531616    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:28.531849 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531703    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:28.531849 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531840    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:50:28.532073 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531918    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:28.532073 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.531622    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-socket\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.532073 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.532001    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-data\") pod \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\" (UID: \"5591bae4-a799-462a-b00d-fe0dff4cdbb3\") "
Sep 10 23:50:28.532299 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.532252    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-data" (OuterVolumeSpecName: "istio-data") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:28.532920 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.532899    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:28.533043 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.532924    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:28.533043 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.532956    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:28.533043 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.532971    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:28.533043 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.532984    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:28.533749 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.533719    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:28.533960 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.533909    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-token" (OuterVolumeSpecName: "istio-token") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:50:28.534033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.533924    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-kube-api-access-dxk85" (OuterVolumeSpecName: "kube-api-access-dxk85") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "kube-api-access-dxk85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:50:28.534033 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.533962    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5591bae4-a799-462a-b00d-fe0dff4cdbb3" (UID: "5591bae4-a799-462a-b00d-fe0dff4cdbb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:50:28.634282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.634234    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:28.634282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.634276    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxk85\" (UniqueName: \"kubernetes.io/projected/5591bae4-a799-462a-b00d-fe0dff4cdbb3-kube-api-access-dxk85\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:28.634282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.634291    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:28.634518 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:28.634304    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5591bae4-a799-462a-b00d-fe0dff4cdbb3-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:29.214554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:29.214511    2569 patch_prober.go:28] interesting pod/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 23:50:29.214757 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:29.214586    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 23:50:29.407555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:29.407523    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"
Sep 10 23:50:29.408051 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:29.407529    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq" event={"ID":"5591bae4-a799-462a-b00d-fe0dff4cdbb3","Type":"ContainerDied","Data":"56ccdf3aae433e5d7f6ee8e1bbbff853b9ab8860e19f24d457d0de0f1d94173f"}
Sep 10 23:50:29.408051 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:29.407651    2569 scope.go:117] "RemoveContainer" containerID="a812cb8ae666cd8c7ba86f006c90252038d2e23cb67faee05252c934a7607ab1"
Sep 10 23:50:29.425736 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:29.425710    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"]
Sep 10 23:50:29.430619 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:29.430598    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-487960-openshift-default-79857f5b6-gmvxq"]
Sep 10 23:50:30.214408 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:30.214369    2569 patch_prober.go:28] interesting pod/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 23:50:30.214564 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:30.214428    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 23:50:30.978662 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:30.978626    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" path="/var/lib/kubelet/pods/5591bae4-a799-462a-b00d-fe0dff4cdbb3/volumes"
Sep 10 23:50:31.214632 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:31.214591    2569 patch_prober.go:28] interesting pod/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 23:50:31.214795 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:31.214647    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 23:50:32.214216 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:32.214172    2569 patch_prober.go:28] interesting pod/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 10 23:50:32.214594 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:32.214245    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 10 23:50:33.253751 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:33.253718    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:33.254213 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:33.254066    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:33.254917 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:33.254896    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:50:34.150335 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.150296    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"]
Sep 10 23:50:34.150573 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.150543    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv" podUID="8339d5a5-f3e8-4bdf-a84f-8d9f6c353394" containerName="payload-processing" containerID="cri-o://992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55" gracePeriod=30
Sep 10 23:50:34.431569 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.431543    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:50:34.433961 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.433918    2569 generic.go:356] "Generic (PLEG): container finished" podID="8339d5a5-f3e8-4bdf-a84f-8d9f6c353394" containerID="992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55" exitCode=0
Sep 10 23:50:34.434084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.433977    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"
Sep 10 23:50:34.434084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.433978    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv" event={"ID":"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394","Type":"ContainerDied","Data":"992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55"}
Sep 10 23:50:34.434084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.434036    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv" event={"ID":"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394","Type":"ContainerDied","Data":"0649b35728d217e90eb88fcb357d8c4a406331e4e21874f07ea8e6981471d432"}
Sep 10 23:50:34.434084 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.434064    2569 scope.go:117] "RemoveContainer" containerID="992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55"
Sep 10 23:50:34.442457 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.442439    2569 scope.go:117] "RemoveContainer" containerID="992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55"
Sep 10 23:50:34.442718 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:50:34.442701    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55\": container with ID starting with 992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55 not found: ID does not exist" containerID="992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55"
Sep 10 23:50:34.442776 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.442729    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55"} err="failed to get container status \"992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55\": rpc error: code = NotFound desc = could not find container \"992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55\": container with ID starting with 992e09e0dadca2274629bdfd19dfd5a23c725bc8a045342776e6769799f73c55 not found: ID does not exist"
Sep 10 23:50:34.599083 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.599046    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-plugins-config-volume\") pod \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\" (UID: \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\") "
Sep 10 23:50:34.599226 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.599189    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97745\" (UniqueName: \"kubernetes.io/projected/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-kube-api-access-97745\") pod \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\" (UID: \"8339d5a5-f3e8-4bdf-a84f-8d9f6c353394\") "
Sep 10 23:50:34.599406 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.599384    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8339d5a5-f3e8-4bdf-a84f-8d9f6c353394" (UID: "8339d5a5-f3e8-4bdf-a84f-8d9f6c353394"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:50:34.599515 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.599496    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:34.601293 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.601270    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-kube-api-access-97745" (OuterVolumeSpecName: "kube-api-access-97745") pod "8339d5a5-f3e8-4bdf-a84f-8d9f6c353394" (UID: "8339d5a5-f3e8-4bdf-a84f-8d9f6c353394"). InnerVolumeSpecName "kube-api-access-97745". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:50:34.700440 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.700349    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97745\" (UniqueName: \"kubernetes.io/projected/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394-kube-api-access-97745\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:34.763365 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.763334    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"]
Sep 10 23:50:34.766735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.766708    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ea66ce-f799774f9-5vbhv"]
Sep 10 23:50:34.978905 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:34.978824    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8339d5a5-f3e8-4bdf-a84f-8d9f6c353394" path="/var/lib/kubelet/pods/8339d5a5-f3e8-4bdf-a84f-8d9f6c353394/volumes"
Sep 10 23:50:40.793126 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:40.793088    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"]
Sep 10 23:50:40.793583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:40.793333    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy" containerID="cri-o://06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da" gracePeriod=30
Sep 10 23:50:41.600473 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.600433    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"]
Sep 10 23:50:41.601032 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601016    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy"
Sep 10 23:50:41.601100 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601036    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy"
Sep 10 23:50:41.601100 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601054    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ed38e3-dfb1-4827-872e-5ecaa74e9c29" containerName="payload-processing"
Sep 10 23:50:41.601100 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601063    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ed38e3-dfb1-4827-872e-5ecaa74e9c29" containerName="payload-processing"
Sep 10 23:50:41.601100 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601073    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8339d5a5-f3e8-4bdf-a84f-8d9f6c353394" containerName="payload-processing"
Sep 10 23:50:41.601100 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601081    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8339d5a5-f3e8-4bdf-a84f-8d9f6c353394" containerName="payload-processing"
Sep 10 23:50:41.601554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601173    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy"
Sep 10 23:50:41.601554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601187    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy"
Sep 10 23:50:41.601554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601197    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8339d5a5-f3e8-4bdf-a84f-8d9f6c353394" containerName="payload-processing"
Sep 10 23:50:41.601554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.601209    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ed38e3-dfb1-4827-872e-5ecaa74e9c29" containerName="payload-processing"
Sep 10 23:50:41.606068 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.606046    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.608845 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.608825    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-0b637b-openshift-default-dockercfg-nvrnv\""
Sep 10 23:50:41.616056 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.616031    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"]
Sep 10 23:50:41.696345 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.696309    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"]
Sep 10 23:50:41.696912 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.696892    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy"
Sep 10 23:50:41.697025 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.696913    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5591bae4-a799-462a-b00d-fe0dff4cdbb3" containerName="istio-proxy"
Sep 10 23:50:41.700470 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.700446    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"
Sep 10 23:50:41.711015 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.709537    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"]
Sep 10 23:50:41.711015 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.710260    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:50:41.711015 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.710832    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:50:41.766232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766194    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-data\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.766409 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqjwn\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-kube-api-access-fqjwn\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.766409 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istiod-ca-cert\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.766409 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-certs\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.766409 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766327    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-socket\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.766409 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766397    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-podinfo\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.766594 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766462    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-envoy\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.766594 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-token\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.766594 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.766549    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-credential-socket\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868057 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-envoy\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-token\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868127    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-credential-socket\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868160    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-data\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqjwn\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-kube-api-access-fqjwn\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istiod-ca-cert\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868257    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-certs\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868293    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-socket\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868341    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-podinfo\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.868549 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868537    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5cf61-cde7-4a3e-974f-962273491ccb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv\" (UID: \"e6b5cf61-cde7-4a3e-974f-962273491ccb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"
Sep 10 23:50:41.869089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868591    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-credential-socket\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.869089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868635    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-data\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.869089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868712    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-certs\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.869089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868862    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-socket\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.869089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.868956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istiod-ca-cert\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.870350 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.870326    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-envoy\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.870883 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.870862    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-podinfo\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.876754 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.876731    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-token\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.876828 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.876790    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqjwn\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-kube-api-access-fqjwn\") pod \"e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") " pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.918656 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.918622    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:41.970372 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.969891    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5cf61-cde7-4a3e-974f-962273491ccb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv\" (UID: \"e6b5cf61-cde7-4a3e-974f-962273491ccb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"
Sep 10 23:50:41.970831 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:41.970804    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5cf61-cde7-4a3e-974f-962273491ccb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv\" (UID: \"e6b5cf61-cde7-4a3e-974f-962273491ccb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"
Sep 10 23:50:42.025662 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.025630    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"
Sep 10 23:50:42.197343 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.197316    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"]
Sep 10 23:50:42.199082 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:50:42.199053    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod132ee4ff_a5ea_4897_a2e2_c246dbb7743c.slice/crio-eeace63a8447515207d0d28a531effbdba14ebeca13f208aaf399aff1016b629 WatchSource:0}: Error finding container eeace63a8447515207d0d28a531effbdba14ebeca13f208aaf399aff1016b629: Status 404 returned error can't find the container with id eeace63a8447515207d0d28a531effbdba14ebeca13f208aaf399aff1016b629
Sep 10 23:50:42.201418 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.201385    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:42.201526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.201462    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:42.201526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.201509    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:50:42.299257 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.299226    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"]
Sep 10 23:50:42.301420 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:50:42.301388    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b5cf61_cde7_4a3e_974f_962273491ccb.slice/crio-de528ccecd7586a62215cbeac7026e9d919a7f8f8b8989d4b8a642dab266b083 WatchSource:0}: Error finding container de528ccecd7586a62215cbeac7026e9d919a7f8f8b8989d4b8a642dab266b083: Status 404 returned error can't find the container with id de528ccecd7586a62215cbeac7026e9d919a7f8f8b8989d4b8a642dab266b083
Sep 10 23:50:42.481848 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.481757    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" event={"ID":"132ee4ff-a5ea-4897-a2e2-c246dbb7743c","Type":"ContainerStarted","Data":"218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a"}
Sep 10 23:50:42.481848 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.481817    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" event={"ID":"132ee4ff-a5ea-4897-a2e2-c246dbb7743c","Type":"ContainerStarted","Data":"eeace63a8447515207d0d28a531effbdba14ebeca13f208aaf399aff1016b629"}
Sep 10 23:50:42.483112 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.483089    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv" event={"ID":"e6b5cf61-cde7-4a3e-974f-962273491ccb","Type":"ContainerStarted","Data":"de528ccecd7586a62215cbeac7026e9d919a7f8f8b8989d4b8a642dab266b083"}
Sep 10 23:50:42.500970 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.500901    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podStartSLOduration=1.5008832239999998 podStartE2EDuration="1.500883224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:42.499663705 +0000 UTC m=+1056.153383892" watchObservedRunningTime="2026-09-10 23:50:42.500883224 +0000 UTC m=+1056.154603398"
Sep 10 23:50:42.919634 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:42.919594    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:43.490439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:43.490398    2569 generic.go:356] "Generic (PLEG): container finished" podID="e6b5cf61-cde7-4a3e-974f-962273491ccb" containerID="357fc7fa47357d769eef844b2a3a9e6c2df29128dc2a0b581b75297d52196e89" exitCode=0
Sep 10 23:50:43.490616 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:43.490487    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv" event={"ID":"e6b5cf61-cde7-4a3e-974f-962273491ccb","Type":"ContainerDied","Data":"357fc7fa47357d769eef844b2a3a9e6c2df29128dc2a0b581b75297d52196e89"}
Sep 10 23:50:43.924323 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:43.924291    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:44.496136 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:44.496102    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:44.497041 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:44.497013    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:50:44.646012 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:44.645989    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"
Sep 10 23:50:44.798529 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:44.798439    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5cf61-cde7-4a3e-974f-962273491ccb-maas-api-service-ca\") pod \"e6b5cf61-cde7-4a3e-974f-962273491ccb\" (UID: \"e6b5cf61-cde7-4a3e-974f-962273491ccb\") "
Sep 10 23:50:44.798843 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:44.798821    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b5cf61-cde7-4a3e-974f-962273491ccb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6b5cf61-cde7-4a3e-974f-962273491ccb" (UID: "e6b5cf61-cde7-4a3e-974f-962273491ccb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:50:44.899746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:44.899710    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b5cf61-cde7-4a3e-974f-962273491ccb-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:45.501948 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:45.501909    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"
Sep 10 23:50:45.502337 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:45.501961    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv" event={"ID":"e6b5cf61-cde7-4a3e-974f-962273491ccb","Type":"ContainerDied","Data":"de528ccecd7586a62215cbeac7026e9d919a7f8f8b8989d4b8a642dab266b083"}
Sep 10 23:50:45.502337 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:45.502007    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de528ccecd7586a62215cbeac7026e9d919a7f8f8b8989d4b8a642dab266b083"
Sep 10 23:50:46.057227 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.057167    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:50:46.213724 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.213686    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-socket\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.213923 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.213744    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-envoy\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.213923 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.213805    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-istio-token\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.213923 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.213827    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-data\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.214149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.213976    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z9d7\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-kube-api-access-4z9d7\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.214149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214013    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:46.214149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214042    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d38d48-20f1-4730-887c-1457b6849524-istiod-ca-cert\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.214149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214094    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d38d48-20f1-4730-887c-1457b6849524-istio-podinfo\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.214149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214120    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-data" (OuterVolumeSpecName: "istio-data") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:46.214149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214130    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-credential-socket\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.214478 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214169    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-certs\") pod \"91d38d48-20f1-4730-887c-1457b6849524\" (UID: \"91d38d48-20f1-4730-887c-1457b6849524\") "
Sep 10 23:50:46.214478 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214390    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:46.214478 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214398    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91d38d48-20f1-4730-887c-1457b6849524-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:50:46.214638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214534    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.214638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214554    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.214638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214569    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91d38d48-20f1-4730-887c-1457b6849524-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.214638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214581    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.214638 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.214587    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:46.216952 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.216263    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-kube-api-access-4z9d7" (OuterVolumeSpecName: "kube-api-access-4z9d7") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "kube-api-access-4z9d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:50:46.216952 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.216737    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91d38d48-20f1-4730-887c-1457b6849524-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:50:46.216952 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.216786    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:50:46.216952 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.216925    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-istio-token" (OuterVolumeSpecName: "istio-token") pod "91d38d48-20f1-4730-887c-1457b6849524" (UID: "91d38d48-20f1-4730-887c-1457b6849524"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:50:46.315746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.315664    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.315746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.315692    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.315746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.315701    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z9d7\" (UniqueName: \"kubernetes.io/projected/91d38d48-20f1-4730-887c-1457b6849524-kube-api-access-4z9d7\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.315746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.315710    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91d38d48-20f1-4730-887c-1457b6849524-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.315746 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.315719    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91d38d48-20f1-4730-887c-1457b6849524-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:46.344040 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344001    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"]
Sep 10 23:50:46.344366 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344355    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy"
Sep 10 23:50:46.344434 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344368    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy"
Sep 10 23:50:46.344434 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344380    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy"
Sep 10 23:50:46.344434 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344385    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy"
Sep 10 23:50:46.344434 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344399    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b5cf61-cde7-4a3e-974f-962273491ccb" containerName="revoke-keys"
Sep 10 23:50:46.344434 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344405    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b5cf61-cde7-4a3e-974f-962273491ccb" containerName="revoke-keys"
Sep 10 23:50:46.344603 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344465    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy"
Sep 10 23:50:46.344603 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.344474    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b5cf61-cde7-4a3e-974f-962273491ccb" containerName="revoke-keys"
Sep 10 23:50:46.347642 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.347626    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:46.349637 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.349613    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-0b637b\""
Sep 10 23:50:46.349759 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.349711    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-0b637b-dockercfg-chzmv\""
Sep 10 23:50:46.359532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.359508    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"]
Sep 10 23:50:46.385507 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.385479    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"]
Sep 10 23:50:46.385909 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.385897    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="91d38d48-20f1-4730-887c-1457b6849524" containerName="istio-proxy"
Sep 10 23:50:46.388866 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.388852    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:46.399114 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.399091    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"]
Sep 10 23:50:46.508761 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.508725    2569 generic.go:356] "Generic (PLEG): container finished" podID="91d38d48-20f1-4730-887c-1457b6849524" containerID="06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da" exitCode=0
Sep 10 23:50:46.509189 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.508773    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"
Sep 10 23:50:46.509189 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.508809    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" event={"ID":"91d38d48-20f1-4730-887c-1457b6849524","Type":"ContainerDied","Data":"06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da"}
Sep 10 23:50:46.509189 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.508860    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv" event={"ID":"91d38d48-20f1-4730-887c-1457b6849524","Type":"ContainerDied","Data":"843be2f5edc08b6ac376be2c4b1eb80ac6cec0205ad11470a1302473df73176b"}
Sep 10 23:50:46.509189 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.508891    2569 scope.go:117] "RemoveContainer" containerID="06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da"
Sep 10 23:50:46.517283 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.517260    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwtbd\" (UniqueName: \"kubernetes.io/projected/60f741ba-7190-43c8-bec7-fcfd06ac79d6-kube-api-access-kwtbd\") pod \"payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf\" (UID: \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:46.517393 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.517305    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60f741ba-7190-43c8-bec7-fcfd06ac79d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf\" (UID: \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:46.517535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.517417    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3454cdc-1a6c-4420-8191-50d59c65e336-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d\" (UID: \"e3454cdc-1a6c-4420-8191-50d59c65e336\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:46.517535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.517477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7dbt\" (UniqueName: \"kubernetes.io/projected/e3454cdc-1a6c-4420-8191-50d59c65e336-kube-api-access-w7dbt\") pod \"payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d\" (UID: \"e3454cdc-1a6c-4420-8191-50d59c65e336\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:46.519024 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.519008    2569 scope.go:117] "RemoveContainer" containerID="52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd"
Sep 10 23:50:46.527247 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.527229    2569 scope.go:117] "RemoveContainer" containerID="06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da"
Sep 10 23:50:46.527519 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:50:46.527502    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da\": container with ID starting with 06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da not found: ID does not exist" containerID="06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da"
Sep 10 23:50:46.527572 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.527529    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da"} err="failed to get container status \"06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da\": rpc error: code = NotFound desc = could not find container \"06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da\": container with ID starting with 06c4a8246ba305e3dccdde5d6b4b88a8a8b56cac451f80daf7d84d8702f688da not found: ID does not exist"
Sep 10 23:50:46.527572 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.527547    2569 scope.go:117] "RemoveContainer" containerID="52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd"
Sep 10 23:50:46.527794 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:50:46.527776    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd\": container with ID starting with 52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd not found: ID does not exist" containerID="52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd"
Sep 10 23:50:46.527861 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.527802    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd"} err="failed to get container status \"52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd\": rpc error: code = NotFound desc = could not find container \"52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd\": container with ID starting with 52d88f6e770d4c681ed9f76d3c39f204c4cc4bdf874cdf86e785656f7c17f4bd not found: ID does not exist"
Sep 10 23:50:46.531077 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.531054    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"]
Sep 10 23:50:46.534286 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.534265    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ea66ce-openshift-default-7cfbbdd45b-5krmv"]
Sep 10 23:50:46.618369 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.618322    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbd\" (UniqueName: \"kubernetes.io/projected/60f741ba-7190-43c8-bec7-fcfd06ac79d6-kube-api-access-kwtbd\") pod \"payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf\" (UID: \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:46.618546 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.618421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60f741ba-7190-43c8-bec7-fcfd06ac79d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf\" (UID: \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:46.618635 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.618601    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3454cdc-1a6c-4420-8191-50d59c65e336-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d\" (UID: \"e3454cdc-1a6c-4420-8191-50d59c65e336\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:46.618695 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.618678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7dbt\" (UniqueName: \"kubernetes.io/projected/e3454cdc-1a6c-4420-8191-50d59c65e336-kube-api-access-w7dbt\") pod \"payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d\" (UID: \"e3454cdc-1a6c-4420-8191-50d59c65e336\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:46.619288 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.619264    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60f741ba-7190-43c8-bec7-fcfd06ac79d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf\" (UID: \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:46.619387 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.619315    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3454cdc-1a6c-4420-8191-50d59c65e336-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d\" (UID: \"e3454cdc-1a6c-4420-8191-50d59c65e336\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:46.626145 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.626124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7dbt\" (UniqueName: \"kubernetes.io/projected/e3454cdc-1a6c-4420-8191-50d59c65e336-kube-api-access-w7dbt\") pod \"payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d\" (UID: \"e3454cdc-1a6c-4420-8191-50d59c65e336\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:46.626238 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.626180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwtbd\" (UniqueName: \"kubernetes.io/projected/60f741ba-7190-43c8-bec7-fcfd06ac79d6-kube-api-access-kwtbd\") pod \"payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf\" (UID: \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:46.659491 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.659464    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:46.700177 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.700143    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:46.932166 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.932140    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"]
Sep 10 23:50:46.934709 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:50:46.934682    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3454cdc_1a6c_4420_8191_50d59c65e336.slice/crio-142a16ba58f50ccdc51e19c494cbe169540e447dddfe1885167afdee2e57294e WatchSource:0}: Error finding container 142a16ba58f50ccdc51e19c494cbe169540e447dddfe1885167afdee2e57294e: Status 404 returned error can't find the container with id 142a16ba58f50ccdc51e19c494cbe169540e447dddfe1885167afdee2e57294e
Sep 10 23:50:46.980627 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.980580    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91d38d48-20f1-4730-887c-1457b6849524" path="/var/lib/kubelet/pods/91d38d48-20f1-4730-887c-1457b6849524/volumes"
Sep 10 23:50:46.981211 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:46.981191    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"]
Sep 10 23:50:47.001128 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:50:47.001100    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f741ba_7190_43c8_bec7_fcfd06ac79d6.slice/crio-f01a461f2b9dbf89d2bf7f27bd0d85a9637894abcfc5c0a20fca45932700368e WatchSource:0}: Error finding container f01a461f2b9dbf89d2bf7f27bd0d85a9637894abcfc5c0a20fca45932700368e: Status 404 returned error can't find the container with id f01a461f2b9dbf89d2bf7f27bd0d85a9637894abcfc5c0a20fca45932700368e
Sep 10 23:50:47.517439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:47.517405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d" event={"ID":"e3454cdc-1a6c-4420-8191-50d59c65e336","Type":"ContainerStarted","Data":"268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90"}
Sep 10 23:50:47.518029 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:47.517453    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d" event={"ID":"e3454cdc-1a6c-4420-8191-50d59c65e336","Type":"ContainerStarted","Data":"142a16ba58f50ccdc51e19c494cbe169540e447dddfe1885167afdee2e57294e"}
Sep 10 23:50:47.518029 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:47.517506    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:47.519224 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:47.519193    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf" event={"ID":"60f741ba-7190-43c8-bec7-fcfd06ac79d6","Type":"ContainerStarted","Data":"610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa"}
Sep 10 23:50:47.519368 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:47.519243    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf" event={"ID":"60f741ba-7190-43c8-bec7-fcfd06ac79d6","Type":"ContainerStarted","Data":"f01a461f2b9dbf89d2bf7f27bd0d85a9637894abcfc5c0a20fca45932700368e"}
Sep 10 23:50:47.519368 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:47.519271    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:47.533929 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:47.533877    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d" podStartSLOduration=1.533860269 podStartE2EDuration="1.533860269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:47.533353612 +0000 UTC m=+1061.187073787" watchObservedRunningTime="2026-09-10 23:50:47.533860269 +0000 UTC m=+1061.187580443"
Sep 10 23:50:47.550521 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:47.550475    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf" podStartSLOduration=1.550461533 podStartE2EDuration="1.550461533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:50:47.549494019 +0000 UTC m=+1061.203214193" watchObservedRunningTime="2026-09-10 23:50:47.550461533 +0000 UTC m=+1061.204181741"
Sep 10 23:50:52.279514 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.279482    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"]
Sep 10 23:50:52.279991 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.279741    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff" podUID="43687a4c-639f-45cd-8179-6998d7bc0afc" containerName="payload-pre-processing" containerID="cri-o://b70c644e4b52a24df39bd47c2217b4f97a302b10d44b2be0d3e00db843ae37ec" gracePeriod=30
Sep 10 23:50:52.413315 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.413279    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"]
Sep 10 23:50:52.416452 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.416414    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e5eaec01-btgmv"]
Sep 10 23:50:52.573924 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.573890    2569 generic.go:356] "Generic (PLEG): container finished" podID="43687a4c-639f-45cd-8179-6998d7bc0afc" containerID="b70c644e4b52a24df39bd47c2217b4f97a302b10d44b2be0d3e00db843ae37ec" exitCode=0
Sep 10 23:50:52.574105 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.573964    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff" event={"ID":"43687a4c-639f-45cd-8179-6998d7bc0afc","Type":"ContainerDied","Data":"b70c644e4b52a24df39bd47c2217b4f97a302b10d44b2be0d3e00db843ae37ec"}
Sep 10 23:50:52.574105 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.574015    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff" event={"ID":"43687a4c-639f-45cd-8179-6998d7bc0afc","Type":"ContainerDied","Data":"d6aa76c4c50502cac06bf634ca44eb559d8e6d53a96633b77dbee5044338387b"}
Sep 10 23:50:52.574105 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.574031    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6aa76c4c50502cac06bf634ca44eb559d8e6d53a96633b77dbee5044338387b"
Sep 10 23:50:52.576317 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.576299    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:52.670421 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.670388    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2mgz\" (UniqueName: \"kubernetes.io/projected/43687a4c-639f-45cd-8179-6998d7bc0afc-kube-api-access-k2mgz\") pod \"43687a4c-639f-45cd-8179-6998d7bc0afc\" (UID: \"43687a4c-639f-45cd-8179-6998d7bc0afc\") "
Sep 10 23:50:52.672479 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.672459    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43687a4c-639f-45cd-8179-6998d7bc0afc-kube-api-access-k2mgz" (OuterVolumeSpecName: "kube-api-access-k2mgz") pod "43687a4c-639f-45cd-8179-6998d7bc0afc" (UID: "43687a4c-639f-45cd-8179-6998d7bc0afc"). InnerVolumeSpecName "kube-api-access-k2mgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:50:52.771584 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.771550    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43687a4c-639f-45cd-8179-6998d7bc0afc-plugins-config-volume\") pod \"43687a4c-639f-45cd-8179-6998d7bc0afc\" (UID: \"43687a4c-639f-45cd-8179-6998d7bc0afc\") "
Sep 10 23:50:52.771787 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.771707    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2mgz\" (UniqueName: \"kubernetes.io/projected/43687a4c-639f-45cd-8179-6998d7bc0afc-kube-api-access-k2mgz\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:52.771892 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.771868    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43687a4c-639f-45cd-8179-6998d7bc0afc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "43687a4c-639f-45cd-8179-6998d7bc0afc" (UID: "43687a4c-639f-45cd-8179-6998d7bc0afc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:50:52.872838 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.872796    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43687a4c-639f-45cd-8179-6998d7bc0afc-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:50:52.978853 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:52.978823    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b5cf61-cde7-4a3e-974f-962273491ccb" path="/var/lib/kubelet/pods/e6b5cf61-cde7-4a3e-974f-962273491ccb/volumes"
Sep 10 23:50:53.579006 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:53.578972    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"
Sep 10 23:50:53.594836 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:53.594806    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"]
Sep 10 23:50:53.598287 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:53.598264    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e5eaec01-6665d6574b-pw8ff"]
Sep 10 23:50:54.978500 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:54.978461    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43687a4c-639f-45cd-8179-6998d7bc0afc" path="/var/lib/kubelet/pods/43687a4c-639f-45cd-8179-6998d7bc0afc/volumes"
Sep 10 23:50:58.526544 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:58.526513    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 10 23:50:58.527002 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:58.526591    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 10 23:50:58.886662 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:58.886620    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"]
Sep 10 23:50:58.886922 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:50:58.886890    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy" containerID="cri-o://f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b" gracePeriod=30
Sep 10 23:51:00.005885 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:00.005137    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:58500->10.132.0.62:15021: read: connection reset by peer" start-of-body=
Sep 10 23:51:00.005885 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:00.005214    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:58500->10.132.0.62:15021: read: connection reset by peer"
Sep 10 23:51:00.617603 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:00.617570    2569 generic.go:356] "Generic (PLEG): container finished" podID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerID="218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a" exitCode=0
Sep 10 23:51:00.617787 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:00.617643    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" event={"ID":"132ee4ff-a5ea-4897-a2e2-c246dbb7743c","Type":"ContainerDied","Data":"218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a"}
Sep 10 23:51:00.618147 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:00.618132    2569 scope.go:117] "RemoveContainer" containerID="218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a"
Sep 10 23:51:00.619258 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:00.619230    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:00.619362 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:00.619317    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:00.619404 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:00.619363    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:01.625541 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:01.625498    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" event={"ID":"132ee4ff-a5ea-4897-a2e2-c246dbb7743c","Type":"ContainerStarted","Data":"d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3"}
Sep 10 23:51:01.918922 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:01.918835    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:51:01.919114 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:01.919088    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 10 23:51:01.919183 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:01.919161    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 10 23:51:02.919597 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:02.919564    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 10 23:51:02.920076 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:02.919621    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 10 23:51:03.258652 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:03.258557    2569 patch_prober.go:28] interesting pod/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 23:51:03.259245 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:03.258919    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 23:51:03.919390 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:03.919365    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 10 23:51:03.919502 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:03.919424    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 10 23:51:04.184161 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.184137    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:51:04.282410 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282377    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kn7b\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-kube-api-access-6kn7b\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.282410 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282416    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-credential-socket\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.282658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282438    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-certs\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.282658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282456    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-socket\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.282658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282483    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-podinfo\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.282658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282531    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-envoy\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.282658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282555    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-token\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.282658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282609    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-data\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.282658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282643    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istiod-ca-cert\") pod \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\" (UID: \"4c45e003-91c4-486c-89a7-3ff56e2a0bbc\") "
Sep 10 23:51:04.283067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282700    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:51:04.283067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282829    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:51:04.283067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.282886    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:51:04.283067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.283044    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.283067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.283064    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.283250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.283078    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.283250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.283099    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:51:04.283250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.283126    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:51:04.284831 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.284807    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:51:04.284960 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.284868    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:51:04.285119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.285087    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:51:04.285217 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.285144    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-kube-api-access-6kn7b" (OuterVolumeSpecName: "kube-api-access-6kn7b") pod "4c45e003-91c4-486c-89a7-3ff56e2a0bbc" (UID: "4c45e003-91c4-486c-89a7-3ff56e2a0bbc"). InnerVolumeSpecName "kube-api-access-6kn7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:51:04.383729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.383691    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.383729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.383723    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kn7b\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-kube-api-access-6kn7b\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.383729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.383734    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.383729 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.383742    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.384039 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.383750    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.384039 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.383758    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c45e003-91c4-486c-89a7-3ff56e2a0bbc-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:51:04.646137 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.646098    2569 generic.go:356] "Generic (PLEG): container finished" podID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerID="f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b" exitCode=0
Sep 10 23:51:04.646410 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.646311    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" event={"ID":"4c45e003-91c4-486c-89a7-3ff56e2a0bbc","Type":"ContainerDied","Data":"f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b"}
Sep 10 23:51:04.646410 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.646381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp" event={"ID":"4c45e003-91c4-486c-89a7-3ff56e2a0bbc","Type":"ContainerDied","Data":"44ddb76d57ea20c163e7405e96173ed4e53fa6bda43134ae8bc3f4bff37b5f5d"}
Sep 10 23:51:04.646540 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.646413    2569 scope.go:117] "RemoveContainer" containerID="f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b"
Sep 10 23:51:04.646600 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.646589    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"
Sep 10 23:51:04.667172 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.667138    2569 scope.go:117] "RemoveContainer" containerID="9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f"
Sep 10 23:51:04.679849 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.679812    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"]
Sep 10 23:51:04.681525 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.681500    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e5eaec01-openshift-default-6448c68cd8-5wrvp"]
Sep 10 23:51:04.685078 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.685048    2569 scope.go:117] "RemoveContainer" containerID="f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b"
Sep 10 23:51:04.685568 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:51:04.685538    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b\": container with ID starting with f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b not found: ID does not exist" containerID="f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b"
Sep 10 23:51:04.685644 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.685598    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b"} err="failed to get container status \"f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b\": rpc error: code = NotFound desc = could not find container \"f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b\": container with ID starting with f58405e42c09c57d84d7299701e696adb0f8bba4f158cce9fcad639832be165b not found: ID does not exist"
Sep 10 23:51:04.685711 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.685649    2569 scope.go:117] "RemoveContainer" containerID="9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f"
Sep 10 23:51:04.686068 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:51:04.686037    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f\": container with ID starting with 9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f not found: ID does not exist" containerID="9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f"
Sep 10 23:51:04.686142 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.686077    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f"} err="failed to get container status \"9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f\": rpc error: code = NotFound desc = could not find container \"9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f\": container with ID starting with 9804d98fab18bb6b9162e08cf63c7fa1a220c2ee0a3faba09e96cc255d7ff66f not found: ID does not exist"
Sep 10 23:51:04.919918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.919839    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 10 23:51:04.919918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.919893    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 10 23:51:04.979152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:04.979122    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" path="/var/lib/kubelet/pods/4c45e003-91c4-486c-89a7-3ff56e2a0bbc/volumes"
Sep 10 23:51:05.919237 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:05.919198    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Sep 10 23:51:05.919623 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:05.919252    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Sep 10 23:51:06.922036 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:06.922008    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:51:06.922453 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:06.922309    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:51:06.923072 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:06.923054    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 10 23:51:09.844233 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.844202    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"]
Sep 10 23:51:09.844789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.844768    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy"
Sep 10 23:51:09.845171 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.844927    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy"
Sep 10 23:51:09.845171 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.845045    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43687a4c-639f-45cd-8179-6998d7bc0afc" containerName="payload-pre-processing"
Sep 10 23:51:09.845171 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.845056    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="43687a4c-639f-45cd-8179-6998d7bc0afc" containerName="payload-pre-processing"
Sep 10 23:51:09.846326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.845178    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy"
Sep 10 23:51:09.846326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.845192    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy"
Sep 10 23:51:09.846326 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.845203    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="43687a4c-639f-45cd-8179-6998d7bc0afc" containerName="payload-pre-processing"
Sep 10 23:51:09.850192 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.850171    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.853292 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.853268    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-6d2e31-openshift-default-dockercfg-9h2hk\""
Sep 10 23:51:09.859948 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.859913    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"]
Sep 10 23:51:09.937808 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.937768    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4cw7\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-kube-api-access-k4cw7\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.937808 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.937817    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-token\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.938133 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.937901    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-envoy\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.938133 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.937961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41546028-449d-41b6-bb75-0e6b3400bdbf-istiod-ca-cert\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.938133 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.937995    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-credential-socket\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.938133 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.938016    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-podinfo\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.938133 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.938045    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-data\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.938389 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.938172    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-socket\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:09.938389 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:09.938197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-certs\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039106 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039070    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-token\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039305 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-envoy\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039305 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039148    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41546028-449d-41b6-bb75-0e6b3400bdbf-istiod-ca-cert\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039305 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039183    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-credential-socket\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039305 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039207    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-podinfo\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039305 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039237    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-data\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039591 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039349    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-socket\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039591 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-certs\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039591 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4cw7\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-kube-api-access-k4cw7\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039753 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039704    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-data\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039753 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039722    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-socket\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.039865 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039760    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-certs\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.040022 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.039995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-credential-socket\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.040363 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.040342    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41546028-449d-41b6-bb75-0e6b3400bdbf-istiod-ca-cert\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.041970 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.041914    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-envoy\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.042317 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.042296    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-podinfo\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.046498 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.046468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-token\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.047166 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.047148    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4cw7\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-kube-api-access-k4cw7\") pod \"e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") " pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.161444 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.161404    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:10.440312 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.440227    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"]
Sep 10 23:51:10.445262 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:51:10.445209    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41546028_449d_41b6_bb75_0e6b3400bdbf.slice/crio-2335413a377d144e5b2ed8aa4835ba6431df059c03a9f2ad46825b130ae59b47 WatchSource:0}: Error finding container 2335413a377d144e5b2ed8aa4835ba6431df059c03a9f2ad46825b130ae59b47: Status 404 returned error can't find the container with id 2335413a377d144e5b2ed8aa4835ba6431df059c03a9f2ad46825b130ae59b47
Sep 10 23:51:10.448221 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.448179    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:10.448315 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.448266    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:10.448315 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.448306    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:10.694229 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.694134    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" event={"ID":"41546028-449d-41b6-bb75-0e6b3400bdbf","Type":"ContainerStarted","Data":"675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1"}
Sep 10 23:51:10.694229 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.694187    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" event={"ID":"41546028-449d-41b6-bb75-0e6b3400bdbf","Type":"ContainerStarted","Data":"2335413a377d144e5b2ed8aa4835ba6431df059c03a9f2ad46825b130ae59b47"}
Sep 10 23:51:10.713277 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:10.713230    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" podStartSLOduration=1.713214392 podStartE2EDuration="1.713214392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:51:10.712490288 +0000 UTC m=+1084.366210464" watchObservedRunningTime="2026-09-10 23:51:10.713214392 +0000 UTC m=+1084.366934561"
Sep 10 23:51:11.162507 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:11.162473    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:11.167944 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:11.167907    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:11.700143 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:11.700116    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:11.701048 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:11.701020    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:12.740367 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.740331    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"]
Sep 10 23:51:12.740928 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.740907    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy"
Sep 10 23:51:12.740928 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.740930    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c45e003-91c4-486c-89a7-3ff56e2a0bbc" containerName="istio-proxy"
Sep 10 23:51:12.744837 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.744815    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.747320 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.747301    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-4e0933-openshift-default-dockercfg-phn8c\""
Sep 10 23:51:12.764102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.764075    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"]
Sep 10 23:51:12.868012 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.867962    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-certs\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.868204 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.868045    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-socket\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.868204 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.868110    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-envoy\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.868345 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.868217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istiod-ca-cert\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.868345 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.868265    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-token\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.868345 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.868290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55qrn\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-kube-api-access-55qrn\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.868497 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.868373    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-credential-socket\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.868497 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.868408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-podinfo\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.868497 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.868438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-data\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969171 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969139    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-envoy\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969347 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969181    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istiod-ca-cert\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969347 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-token\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969347 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969219    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55qrn\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-kube-api-access-55qrn\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969347 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969305    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-credential-socket\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969347 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969337    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-podinfo\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969622 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969360    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-data\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969622 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969386    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-certs\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969622 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-socket\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.969778 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.969683    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-credential-socket\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.970028 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.970001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-data\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.970166 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.970049    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istiod-ca-cert\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.970166 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.970086    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-socket\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.970284 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.970229    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-certs\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.972065 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.972041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-envoy\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.972356 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.972337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-podinfo\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.977292 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.977271    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-token\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:12.977396 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:12.977362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55qrn\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-kube-api-access-55qrn\") pod \"e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") " pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:13.059158 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:13.059066    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:13.335166 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:13.335129    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"]
Sep 10 23:51:13.338162 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:51:13.338131    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ae0aea_b104_43e7_8435_f9087ec99fb4.slice/crio-54d0699b2d47c6b21f91649905d46fcfd6b1c5d9afec6b5e512da7ac4507432d WatchSource:0}: Error finding container 54d0699b2d47c6b21f91649905d46fcfd6b1c5d9afec6b5e512da7ac4507432d: Status 404 returned error can't find the container with id 54d0699b2d47c6b21f91649905d46fcfd6b1c5d9afec6b5e512da7ac4507432d
Sep 10 23:51:13.340781 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:13.340743    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:13.340875 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:13.340816    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:13.340875 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:13.340845    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:13.713098 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:13.713056    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" event={"ID":"c0ae0aea-b104-43e7-8435-f9087ec99fb4","Type":"ContainerStarted","Data":"12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99"}
Sep 10 23:51:13.713273 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:13.713111    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" event={"ID":"c0ae0aea-b104-43e7-8435-f9087ec99fb4","Type":"ContainerStarted","Data":"54d0699b2d47c6b21f91649905d46fcfd6b1c5d9afec6b5e512da7ac4507432d"}
Sep 10 23:51:13.733092 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:13.733045    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podStartSLOduration=1.7330273109999998 podStartE2EDuration="1.733027311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:51:13.731100775 +0000 UTC m=+1087.384820950" watchObservedRunningTime="2026-09-10 23:51:13.733027311 +0000 UTC m=+1087.386747545"
Sep 10 23:51:14.059551 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:14.059465    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:14.065044 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:14.065018    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:14.718632 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:14.718603    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:14.719613 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:14.719592    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:17.506827 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.506784    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"]
Sep 10 23:51:17.511526 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.511502    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:17.513436 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.513413    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-4e0933\""
Sep 10 23:51:17.522200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.522173    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"]
Sep 10 23:51:17.556877 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.556846    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"]
Sep 10 23:51:17.560354 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.560337    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:17.563560 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.563540    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-4e0933-dockercfg-8fsjz\""
Sep 10 23:51:17.572456 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.572433    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"]
Sep 10 23:51:17.613681 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.613639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nthz8\" (UniqueName: \"kubernetes.io/projected/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-kube-api-access-nthz8\") pod \"payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh\" (UID: \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:17.613875 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.613709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a14ffa32-7d12-4122-ad2d-c6b1398765b1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp\" (UID: \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:17.613875 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.613819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh\" (UID: \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:17.613875 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.613853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mhpn\" (UniqueName: \"kubernetes.io/projected/a14ffa32-7d12-4122-ad2d-c6b1398765b1-kube-api-access-2mhpn\") pod \"payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp\" (UID: \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:17.714754 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.714701    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a14ffa32-7d12-4122-ad2d-c6b1398765b1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp\" (UID: \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:17.714975 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.714788    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh\" (UID: \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:17.714975 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.714812    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mhpn\" (UniqueName: \"kubernetes.io/projected/a14ffa32-7d12-4122-ad2d-c6b1398765b1-kube-api-access-2mhpn\") pod \"payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp\" (UID: \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:17.714975 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.714896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nthz8\" (UniqueName: \"kubernetes.io/projected/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-kube-api-access-nthz8\") pod \"payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh\" (UID: \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:17.715483 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.715462    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a14ffa32-7d12-4122-ad2d-c6b1398765b1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp\" (UID: \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:17.715544 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.715526    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh\" (UID: \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:17.722426 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.722396    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mhpn\" (UniqueName: \"kubernetes.io/projected/a14ffa32-7d12-4122-ad2d-c6b1398765b1-kube-api-access-2mhpn\") pod \"payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp\" (UID: \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:17.722516 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.722405    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nthz8\" (UniqueName: \"kubernetes.io/projected/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-kube-api-access-nthz8\") pod \"payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh\" (UID: \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:17.823805 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.823701    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:17.870775 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:17.870740    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:18.099893 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.099869    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"]
Sep 10 23:51:18.101642 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:51:18.101615    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6438f0_ef81_4b4b_94b5_d21f7e3b295f.slice/crio-a81315152664ec1d90ed805594c6c4a079c7286198f039fb8f13902db5f020ca WatchSource:0}: Error finding container a81315152664ec1d90ed805594c6c4a079c7286198f039fb8f13902db5f020ca: Status 404 returned error can't find the container with id a81315152664ec1d90ed805594c6c4a079c7286198f039fb8f13902db5f020ca
Sep 10 23:51:18.151292 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.151271    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"]
Sep 10 23:51:18.153733 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:51:18.153705    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda14ffa32_7d12_4122_ad2d_c6b1398765b1.slice/crio-6fd83924a70c01a460b8b3eda1b543e095fcf2e4e72f671b458958579c72d5e3 WatchSource:0}: Error finding container 6fd83924a70c01a460b8b3eda1b543e095fcf2e4e72f671b458958579c72d5e3: Status 404 returned error can't find the container with id 6fd83924a70c01a460b8b3eda1b543e095fcf2e4e72f671b458958579c72d5e3
Sep 10 23:51:18.743452 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.743417    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp" event={"ID":"a14ffa32-7d12-4122-ad2d-c6b1398765b1","Type":"ContainerStarted","Data":"fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b"}
Sep 10 23:51:18.744050 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.743479    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp" event={"ID":"a14ffa32-7d12-4122-ad2d-c6b1398765b1","Type":"ContainerStarted","Data":"6fd83924a70c01a460b8b3eda1b543e095fcf2e4e72f671b458958579c72d5e3"}
Sep 10 23:51:18.744050 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.743625    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:18.745359 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.745336    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh" event={"ID":"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f","Type":"ContainerStarted","Data":"5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1"}
Sep 10 23:51:18.745469 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.745381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh" event={"ID":"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f","Type":"ContainerStarted","Data":"a81315152664ec1d90ed805594c6c4a079c7286198f039fb8f13902db5f020ca"}
Sep 10 23:51:18.745469 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.745410    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:18.762550 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.762495    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp" podStartSLOduration=1.7624793859999999 podStartE2EDuration="1.762479386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:51:18.760154704 +0000 UTC m=+1092.413874878" watchObservedRunningTime="2026-09-10 23:51:18.762479386 +0000 UTC m=+1092.416199559"
Sep 10 23:51:18.782682 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:18.782630    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh" podStartSLOduration=1.782614711 podStartE2EDuration="1.782614711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:51:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:51:18.78009152 +0000 UTC m=+1092.433811704" watchObservedRunningTime="2026-09-10 23:51:18.782614711 +0000 UTC m=+1092.436334885"
Sep 10 23:51:29.753659 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:29.753628    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:51:29.754154 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:29.753708    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:51:30.495319 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.495276    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": read tcp 10.132.0.2:60946->10.132.0.67:15021: read: connection reset by peer" start-of-body=
Sep 10 23:51:30.495472 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.495366    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": read tcp 10.132.0.2:60946->10.132.0.67:15021: read: connection reset by peer"
Sep 10 23:51:30.828462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.828384    2569 generic.go:356] "Generic (PLEG): container finished" podID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerID="675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1" exitCode=0
Sep 10 23:51:30.828846 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.828456    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" event={"ID":"41546028-449d-41b6-bb75-0e6b3400bdbf","Type":"ContainerDied","Data":"675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1"}
Sep 10 23:51:30.828968 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.828926    2569 scope.go:117] "RemoveContainer" containerID="675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1"
Sep 10 23:51:30.830116 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.830084    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:30.830237 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.830123    2569 generic.go:356] "Generic (PLEG): container finished" podID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerID="12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99" exitCode=0
Sep 10 23:51:30.830237 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.830169    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:30.830237 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.830189    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" event={"ID":"c0ae0aea-b104-43e7-8435-f9087ec99fb4","Type":"ContainerDied","Data":"12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99"}
Sep 10 23:51:30.830237 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.830203    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:30.830522 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.830509    2569 scope.go:117] "RemoveContainer" containerID="12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99"
Sep 10 23:51:30.831534 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.831509    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:30.831626 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.831582    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:30.831691 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:30.831630    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:51:31.838091 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:31.838050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" event={"ID":"41546028-449d-41b6-bb75-0e6b3400bdbf","Type":"ContainerStarted","Data":"46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f"}
Sep 10 23:51:31.841900 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:31.841829    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" event={"ID":"c0ae0aea-b104-43e7-8435-f9087ec99fb4","Type":"ContainerStarted","Data":"2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868"}
Sep 10 23:51:32.059843 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:32.059800    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:32.060081 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:32.059859    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:51:32.060081 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:32.059921    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:51:32.162264 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:32.162230    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:32.162637 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:32.162536    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:51:32.162816 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:32.162794    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:51:33.059947 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:33.059881    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:51:33.060482 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:33.060001    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:51:33.163506 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:33.163461    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:51:33.163705 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:33.163545    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:51:34.060480 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:34.060439    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:51:34.061119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:34.061082    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:51:34.162611 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:34.162572    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:51:34.162919 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:34.162889    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:51:35.059735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:35.059696    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:51:35.060066 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:35.060038    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:51:35.162266 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:35.162223    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:51:35.162758 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:35.162304    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:51:36.059597 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:36.059548    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 10 23:51:36.059818 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:36.059623    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 10 23:51:36.162723 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:36.162673    2569 patch_prober.go:28] interesting pod/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 10 23:51:36.163247 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:36.162760    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 10 23:51:37.063514 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:37.063478    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:37.063855 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:37.063837    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:37.064667 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:37.064647    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:51:37.172157 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:37.172125    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:37.172992 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:37.172917    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:37.173799 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:37.173729    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 10 23:51:45.643377 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.643340    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"]
Sep 10 23:51:45.647166 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.647148    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:45.650378 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.650356    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-57be92\""
Sep 10 23:51:45.650713 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.650696    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-57be92-dockercfg-2xzdb\""
Sep 10 23:51:45.657887 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.657869    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"]
Sep 10 23:51:45.710165 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.710127    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v65qt\" (UniqueName: \"kubernetes.io/projected/c72380a7-362a-4582-a4e9-11d4d8189ad9-kube-api-access-v65qt\") pod \"payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9\" (UID: \"c72380a7-362a-4582-a4e9-11d4d8189ad9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:45.710356 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.710191    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c72380a7-362a-4582-a4e9-11d4d8189ad9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9\" (UID: \"c72380a7-362a-4582-a4e9-11d4d8189ad9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:45.811692 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.811658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v65qt\" (UniqueName: \"kubernetes.io/projected/c72380a7-362a-4582-a4e9-11d4d8189ad9-kube-api-access-v65qt\") pod \"payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9\" (UID: \"c72380a7-362a-4582-a4e9-11d4d8189ad9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:45.811871 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.811727    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c72380a7-362a-4582-a4e9-11d4d8189ad9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9\" (UID: \"c72380a7-362a-4582-a4e9-11d4d8189ad9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:45.812447 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.812428    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c72380a7-362a-4582-a4e9-11d4d8189ad9-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9\" (UID: \"c72380a7-362a-4582-a4e9-11d4d8189ad9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:45.819449 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.819422    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v65qt\" (UniqueName: \"kubernetes.io/projected/c72380a7-362a-4582-a4e9-11d4d8189ad9-kube-api-access-v65qt\") pod \"payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9\" (UID: \"c72380a7-362a-4582-a4e9-11d4d8189ad9\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:45.957628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:45.957540    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:46.230065 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:46.229980    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"]
Sep 10 23:51:46.233656 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:51:46.233631    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72380a7_362a_4582_a4e9_11d4d8189ad9.slice/crio-d605c2810d590f08ab0b4150375173152382fd62df97ebdc88a7b8bf192925db WatchSource:0}: Error finding container d605c2810d590f08ab0b4150375173152382fd62df97ebdc88a7b8bf192925db: Status 404 returned error can't find the container with id d605c2810d590f08ab0b4150375173152382fd62df97ebdc88a7b8bf192925db
Sep 10 23:51:46.943451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:46.943414    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9" event={"ID":"c72380a7-362a-4582-a4e9-11d4d8189ad9","Type":"ContainerStarted","Data":"0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8"}
Sep 10 23:51:46.943925 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:46.943472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9" event={"ID":"c72380a7-362a-4582-a4e9-11d4d8189ad9","Type":"ContainerStarted","Data":"d605c2810d590f08ab0b4150375173152382fd62df97ebdc88a7b8bf192925db"}
Sep 10 23:51:46.943925 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:46.943556    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:51:46.960113 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:46.960060    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9" podStartSLOduration=1.960044831 podStartE2EDuration="1.960044831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:51:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:51:46.958557664 +0000 UTC m=+1120.612277837" watchObservedRunningTime="2026-09-10 23:51:46.960044831 +0000 UTC m=+1120.613765003"
Sep 10 23:51:57.952808 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:51:57.952775    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:52:18.678792 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.678761    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"]
Sep 10 23:52:18.682399 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.682379    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.684555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.684535    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-42591aef-openshift-default-dockercfg-pgnx2\""
Sep 10 23:52:18.693859 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.693830    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"]
Sep 10 23:52:18.739945 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.739902    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istiod-ca-cert\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.740152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.740004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-envoy\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.740152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.740042    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-podinfo\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.740152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.740071    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-credential-socket\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.740152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.740095    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4bns\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-kube-api-access-h4bns\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.740152 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.740132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-data\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.740443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.740199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-token\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.740443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.740278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-certs\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.740443 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.740345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-socket\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.841821 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.841773    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-data\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.841821 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.841830    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-token\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842088 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.841872    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-certs\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842088 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.841901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-socket\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842088 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842012    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istiod-ca-cert\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842088 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-envoy\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842088 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842077    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-podinfo\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842254 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-credential-socket\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842254 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4bns\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-kube-api-access-h4bns\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842254 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842244    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-data\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842355 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842291    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-socket\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842671 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842649    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-credential-socket\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842861 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842838    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istiod-ca-cert\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.842993 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.842878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-certs\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.844468 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.844448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-envoy\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.844717 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.844700    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-podinfo\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.849448 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.849424    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-token\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.849653 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.849635    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4bns\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-kube-api-access-h4bns\") pod \"e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") " pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:18.994392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:18.994314    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:19.272237 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:19.269995    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"]
Sep 10 23:52:19.278092 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:19.278052    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:52:19.278200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:19.278136    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:52:19.278200 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:19.278195    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:52:20.132046 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:20.132009    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" event={"ID":"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84","Type":"ContainerStarted","Data":"493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c"}
Sep 10 23:52:20.132480 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:20.132059    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" event={"ID":"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84","Type":"ContainerStarted","Data":"04035f8a86acb847e37877966b9f801b8d420a6c91be306188423f8a93556082"}
Sep 10 23:52:20.151379 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:20.151329    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" podStartSLOduration=2.151313746 podStartE2EDuration="2.151313746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:52:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:52:20.148767947 +0000 UTC m=+1153.802488119" watchObservedRunningTime="2026-09-10 23:52:20.151313746 +0000 UTC m=+1153.805033985"
Sep 10 23:52:20.994507 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:20.994467    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:21.000176 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:21.000151    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:21.137394 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:21.137354    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:21.138373 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:21.138350    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:23.901464 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:23.898299    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"]
Sep 10 23:52:23.904498 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:23.903806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:23.907770 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:23.907744    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-42591aef-dockercfg-c5d4k\""
Sep 10 23:52:23.907923 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:23.907756    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-42591aef\""
Sep 10 23:52:23.920988 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:23.920962    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"]
Sep 10 23:52:24.004904 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:24.004862    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws2g7\" (UniqueName: \"kubernetes.io/projected/981b2de9-2eb6-47a5-a535-d40c85ff3e51-kube-api-access-ws2g7\") pod \"payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2\" (UID: \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:24.005115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:24.004926    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981b2de9-2eb6-47a5-a535-d40c85ff3e51-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2\" (UID: \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:24.106390 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:24.106351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws2g7\" (UniqueName: \"kubernetes.io/projected/981b2de9-2eb6-47a5-a535-d40c85ff3e51-kube-api-access-ws2g7\") pod \"payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2\" (UID: \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:24.106571 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:24.106555    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981b2de9-2eb6-47a5-a535-d40c85ff3e51-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2\" (UID: \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:24.107239 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:24.107220    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981b2de9-2eb6-47a5-a535-d40c85ff3e51-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2\" (UID: \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:24.113688 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:24.113659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws2g7\" (UniqueName: \"kubernetes.io/projected/981b2de9-2eb6-47a5-a535-d40c85ff3e51-kube-api-access-ws2g7\") pod \"payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2\" (UID: \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:24.215740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:24.215649    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:24.487484 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:24.487458    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"]
Sep 10 23:52:24.490009 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:52:24.489982    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod981b2de9_2eb6_47a5_a535_d40c85ff3e51.slice/crio-98c8eeb0fb7c43c9a229cc9358453c2351f924971937310ebb129658a480ba1a WatchSource:0}: Error finding container 98c8eeb0fb7c43c9a229cc9358453c2351f924971937310ebb129658a480ba1a: Status 404 returned error can't find the container with id 98c8eeb0fb7c43c9a229cc9358453c2351f924971937310ebb129658a480ba1a
Sep 10 23:52:25.161558 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:25.161520    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2" event={"ID":"981b2de9-2eb6-47a5-a535-d40c85ff3e51","Type":"ContainerStarted","Data":"ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7"}
Sep 10 23:52:25.161970 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:25.161578    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2" event={"ID":"981b2de9-2eb6-47a5-a535-d40c85ff3e51","Type":"ContainerStarted","Data":"98c8eeb0fb7c43c9a229cc9358453c2351f924971937310ebb129658a480ba1a"}
Sep 10 23:52:25.161970 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:25.161641    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:25.179164 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:25.179116    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2" podStartSLOduration=2.179098881 podStartE2EDuration="2.179098881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:52:25.178389376 +0000 UTC m=+1158.832109548" watchObservedRunningTime="2026-09-10 23:52:25.179098881 +0000 UTC m=+1158.832819054"
Sep 10 23:52:36.169495 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:36.169417    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:52:42.265884 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:42.265802    2569 generic.go:356] "Generic (PLEG): container finished" podID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerID="493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c" exitCode=0
Sep 10 23:52:42.265884 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:42.265871    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" event={"ID":"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84","Type":"ContainerDied","Data":"493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c"}
Sep 10 23:52:42.266362 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:42.266292    2569 scope.go:117] "RemoveContainer" containerID="493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c"
Sep 10 23:52:42.267504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:42.267480    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:52:42.267576 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:42.267548    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:52:42.267612 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:42.267576    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:52:43.274001 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:43.273966    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" event={"ID":"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84","Type":"ContainerStarted","Data":"b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd"}
Sep 10 23:52:43.995341 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:43.995300    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:43.995583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:43.995559    2569 patch_prober.go:28] interesting pod/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 10 23:52:43.995630 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:43.995611    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 10 23:52:44.995301 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:44.995267    2569 patch_prober.go:28] interesting pod/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 10 23:52:44.995705 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:44.995320    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 10 23:52:45.995147 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:45.995110    2569 patch_prober.go:28] interesting pod/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 10 23:52:45.995336 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:45.995167    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 10 23:52:46.994776 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:46.994739    2569 patch_prober.go:28] interesting pod/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 10 23:52:46.995010 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:46.994797    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 10 23:52:47.995442 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:47.995407    2569 patch_prober.go:28] interesting pod/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 10 23:52:47.995832 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:47.995461    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 10 23:52:48.998908 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:48.998879    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:48.999304 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:48.999215    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:52:49.000044 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:52:49.000024    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:53:07.059982 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:07.059928    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:53:07.063146 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:07.063125    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:53:07.066527 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:07.066504    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:53:07.069516 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:07.069491    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:53:31.316198 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.316163    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"]
Sep 10 23:53:31.319921 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.319904    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"
Sep 10 23:53:31.321913 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.321892    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:53:31.321913 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.321903    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:53:31.326480 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.326454    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"]
Sep 10 23:53:31.399493 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.399462    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42591aef-zdpwc\" (UID: \"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"
Sep 10 23:53:31.500461 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.500430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42591aef-zdpwc\" (UID: \"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"
Sep 10 23:53:31.501135 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.501110    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-42591aef-zdpwc\" (UID: \"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"
Sep 10 23:53:31.632228 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.632201    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"
Sep 10 23:53:31.902553 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:31.902526    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"]
Sep 10 23:53:31.904547 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:53:31.904511    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9dfdf9f_00d2_42d8_8c49_8eb95cbbaf0c.slice/crio-c9d33553f2601a45e44024a8b672e1625e69a916131612e346e3d76fa1ea864d WatchSource:0}: Error finding container c9d33553f2601a45e44024a8b672e1625e69a916131612e346e3d76fa1ea864d: Status 404 returned error can't find the container with id c9d33553f2601a45e44024a8b672e1625e69a916131612e346e3d76fa1ea864d
Sep 10 23:53:32.546786 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:32.546693    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc" event={"ID":"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c","Type":"ContainerStarted","Data":"76bcc70f737e8e7a6559bb576e031d106c50b96e4bfd20be5c78fb446ba7c81f"}
Sep 10 23:53:32.546786 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:32.546752    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc" event={"ID":"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c","Type":"ContainerStarted","Data":"c9d33553f2601a45e44024a8b672e1625e69a916131612e346e3d76fa1ea864d"}
Sep 10 23:53:32.563143 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:32.563098    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc" podStartSLOduration=1.563083685 podStartE2EDuration="1.563083685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:53:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:53:32.561121695 +0000 UTC m=+1226.214841868" watchObservedRunningTime="2026-09-10 23:53:32.563083685 +0000 UTC m=+1226.216803855"
Sep 10 23:53:33.554371 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:33.554327    2569 generic.go:356] "Generic (PLEG): container finished" podID="b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c" containerID="76bcc70f737e8e7a6559bb576e031d106c50b96e4bfd20be5c78fb446ba7c81f" exitCode=0
Sep 10 23:53:33.554749 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:33.554380    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc" event={"ID":"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c","Type":"ContainerDied","Data":"76bcc70f737e8e7a6559bb576e031d106c50b96e4bfd20be5c78fb446ba7c81f"}
Sep 10 23:53:34.701909 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:34.701885    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"
Sep 10 23:53:34.834642 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:34.834556    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c-maas-api-service-ca\") pod \"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c\" (UID: \"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c\") "
Sep 10 23:53:34.834898 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:34.834877    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c" (UID: "b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:53:34.935890 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:34.935859    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:53:35.567560 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:35.567526    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"
Sep 10 23:53:35.567735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:35.567534    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc" event={"ID":"b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c","Type":"ContainerDied","Data":"c9d33553f2601a45e44024a8b672e1625e69a916131612e346e3d76fa1ea864d"}
Sep 10 23:53:35.567735 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:35.567648    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9d33553f2601a45e44024a8b672e1625e69a916131612e346e3d76fa1ea864d"
Sep 10 23:53:46.793984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:46.793946    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"]
Sep 10 23:53:46.794504 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:46.794219    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2" podUID="981b2de9-2eb6-47a5-a535-d40c85ff3e51" containerName="payload-pre-processing" containerID="cri-o://ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7" gracePeriod=30
Sep 10 23:53:46.944739 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:46.944699    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"]
Sep 10 23:53:46.947070 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:46.947044    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-42591aef-zdpwc"]
Sep 10 23:53:46.980280 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:46.980242    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c" path="/var/lib/kubelet/pods/b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c/volumes"
Sep 10 23:53:47.083960 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.083913    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:53:47.157034 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.157005    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws2g7\" (UniqueName: \"kubernetes.io/projected/981b2de9-2eb6-47a5-a535-d40c85ff3e51-kube-api-access-ws2g7\") pod \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\" (UID: \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\") "
Sep 10 23:53:47.157203 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.157163    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981b2de9-2eb6-47a5-a535-d40c85ff3e51-plugins-config-volume\") pod \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\" (UID: \"981b2de9-2eb6-47a5-a535-d40c85ff3e51\") "
Sep 10 23:53:47.157547 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.157520    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/981b2de9-2eb6-47a5-a535-d40c85ff3e51-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "981b2de9-2eb6-47a5-a535-d40c85ff3e51" (UID: "981b2de9-2eb6-47a5-a535-d40c85ff3e51"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:53:47.159094 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.159071    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981b2de9-2eb6-47a5-a535-d40c85ff3e51-kube-api-access-ws2g7" (OuterVolumeSpecName: "kube-api-access-ws2g7") pod "981b2de9-2eb6-47a5-a535-d40c85ff3e51" (UID: "981b2de9-2eb6-47a5-a535-d40c85ff3e51"). InnerVolumeSpecName "kube-api-access-ws2g7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:53:47.258627 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.258594    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/981b2de9-2eb6-47a5-a535-d40c85ff3e51-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:53:47.258627 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.258622    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws2g7\" (UniqueName: \"kubernetes.io/projected/981b2de9-2eb6-47a5-a535-d40c85ff3e51-kube-api-access-ws2g7\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:53:47.636745 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.636711    2569 generic.go:356] "Generic (PLEG): container finished" podID="981b2de9-2eb6-47a5-a535-d40c85ff3e51" containerID="ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7" exitCode=0
Sep 10 23:53:47.636929 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.636769    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"
Sep 10 23:53:47.636929 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.636798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2" event={"ID":"981b2de9-2eb6-47a5-a535-d40c85ff3e51","Type":"ContainerDied","Data":"ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7"}
Sep 10 23:53:47.636929 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.636852    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2" event={"ID":"981b2de9-2eb6-47a5-a535-d40c85ff3e51","Type":"ContainerDied","Data":"98c8eeb0fb7c43c9a229cc9358453c2351f924971937310ebb129658a480ba1a"}
Sep 10 23:53:47.636929 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.636875    2569 scope.go:117] "RemoveContainer" containerID="ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7"
Sep 10 23:53:47.649162 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.649142    2569 scope.go:117] "RemoveContainer" containerID="ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7"
Sep 10 23:53:47.649476 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:53:47.649456    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7\": container with ID starting with ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7 not found: ID does not exist" containerID="ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7"
Sep 10 23:53:47.649565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.649485    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7"} err="failed to get container status \"ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7\": rpc error: code = NotFound desc = could not find container \"ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7\": container with ID starting with ce1583ec75ebd487ca4ef0d7fb0dcb0ea4b9b7f67ee64c32783e8192a502d1f7 not found: ID does not exist"
Sep 10 23:53:47.660887 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.660856    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"]
Sep 10 23:53:47.663980 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:47.663955    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42591aef-7fcd9b69d6-d6kb2"]
Sep 10 23:53:48.979038 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:53:48.979001    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="981b2de9-2eb6-47a5-a535-d40c85ff3e51" path="/var/lib/kubelet/pods/981b2de9-2eb6-47a5-a535-d40c85ff3e51/volumes"
Sep 10 23:54:07.156144 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:07.156110    2569 scope.go:117] "RemoveContainer" containerID="8c5f363691845129684bdb2f2149e5d939e3a3e43d4a2a41157e40b7a1c332cf"
Sep 10 23:54:24.171727 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:24.171694    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"]
Sep 10 23:54:24.172417 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:24.171908    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy" containerID="cri-o://b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd" gracePeriod=30
Sep 10 23:54:29.432970 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.432925    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:54:29.594225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594119    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-data\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594225 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594206    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istiod-ca-cert\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594260    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-certs\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594291    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4bns\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-kube-api-access-h4bns\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594321    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-credential-socket\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594353    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-podinfo\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594405    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-envoy\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594430    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-token\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594444    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-data" (OuterVolumeSpecName: "istio-data") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:29.594740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594516    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-socket\") pod \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\" (UID: \"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84\") "
Sep 10 23:54:29.594740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594578    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:29.594740 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594660    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:54:29.594981 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594843    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:29.594981 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594895    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.594981 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594912    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.594981 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.594921    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.595155 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.595021    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:29.596838 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.596807    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-token" (OuterVolumeSpecName: "istio-token") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:54:29.596988 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.596835    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:54:29.596988 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.596915    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:54:29.596988 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.596923    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-kube-api-access-h4bns" (OuterVolumeSpecName: "kube-api-access-h4bns") pod "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" (UID: "f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84"). InnerVolumeSpecName "kube-api-access-h4bns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:54:29.696148 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.696117    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.696148 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.696142    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4bns\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-kube-api-access-h4bns\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.696148 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.696153    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.696384 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.696162    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.696384 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.696170    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.696384 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.696179    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:54:29.873798 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.873767    2569 generic.go:356] "Generic (PLEG): container finished" podID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerID="b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd" exitCode=0
Sep 10 23:54:29.874048 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.873813    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"
Sep 10 23:54:29.874048 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.873844    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" event={"ID":"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84","Type":"ContainerDied","Data":"b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd"}
Sep 10 23:54:29.874048 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.873892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5" event={"ID":"f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84","Type":"ContainerDied","Data":"04035f8a86acb847e37877966b9f801b8d420a6c91be306188423f8a93556082"}
Sep 10 23:54:29.874048 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.873915    2569 scope.go:117] "RemoveContainer" containerID="b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd"
Sep 10 23:54:29.883437 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.883421    2569 scope.go:117] "RemoveContainer" containerID="493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c"
Sep 10 23:54:29.891887 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.891870    2569 scope.go:117] "RemoveContainer" containerID="b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd"
Sep 10 23:54:29.892359 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:54:29.892338    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd\": container with ID starting with b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd not found: ID does not exist" containerID="b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd"
Sep 10 23:54:29.892475 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.892369    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd"} err="failed to get container status \"b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd\": rpc error: code = NotFound desc = could not find container \"b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd\": container with ID starting with b07e2204727d8f3f40a610a2aa109f472533ac62625019d9f726bb15613b0ddd not found: ID does not exist"
Sep 10 23:54:29.892475 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.892392    2569 scope.go:117] "RemoveContainer" containerID="493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c"
Sep 10 23:54:29.892903 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:54:29.892783    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c\": container with ID starting with 493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c not found: ID does not exist" containerID="493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c"
Sep 10 23:54:29.892903 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.892817    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c"} err="failed to get container status \"493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c\": rpc error: code = NotFound desc = could not find container \"493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c\": container with ID starting with 493a4b9ac9d9d28f062b0137b86246d5e29f59083834e336594ec580567f377c not found: ID does not exist"
Sep 10 23:54:29.894782 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.894763    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"]
Sep 10 23:54:29.899705 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:29.899684    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-42591aef-openshift-default-5855ffd856-5cgm5"]
Sep 10 23:54:30.978608 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:54:30.978568    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" path="/var/lib/kubelet/pods/f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84/volumes"
Sep 10 23:55:07.184824 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:07.184789    2569 scope.go:117] "RemoveContainer" containerID="0358fbba6171b3a9935b4eb1420e2bd726172428a9b66ce04d5fef75dbcf9517"
Sep 10 23:55:23.164402 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164369    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"]
Sep 10 23:55:23.164868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164773    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy"
Sep 10 23:55:23.164868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164784    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy"
Sep 10 23:55:23.164868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164804    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy"
Sep 10 23:55:23.164868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164810    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy"
Sep 10 23:55:23.164868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164816    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="981b2de9-2eb6-47a5-a535-d40c85ff3e51" containerName="payload-pre-processing"
Sep 10 23:55:23.164868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164821    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="981b2de9-2eb6-47a5-a535-d40c85ff3e51" containerName="payload-pre-processing"
Sep 10 23:55:23.164868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164828    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c" containerName="revoke-keys"
Sep 10 23:55:23.164868 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164833    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c" containerName="revoke-keys"
Sep 10 23:55:23.165232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164897    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy"
Sep 10 23:55:23.165232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164905    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="981b2de9-2eb6-47a5-a535-d40c85ff3e51" containerName="payload-pre-processing"
Sep 10 23:55:23.165232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.164914    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9dfdf9f-00d2-42d8-8c49-8eb95cbbaf0c" containerName="revoke-keys"
Sep 10 23:55:23.169567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.169548    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:23.171496 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.171478    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-7c0647\""
Sep 10 23:55:23.176232 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.176209    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"]
Sep 10 23:55:23.186791 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.186764    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"]
Sep 10 23:55:23.187254 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.187237    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f29f2dd6-6f43-4ea5-9190-3e8b1ddc9f84" containerName="istio-proxy"
Sep 10 23:55:23.190429 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.190407    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:23.192477 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.192457    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-7c0647-dockercfg-vn6dh\""
Sep 10 23:55:23.202370 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.202349    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"]
Sep 10 23:55:23.317507 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.317467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r\" (UID: \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:23.317684 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.317519    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8\" (UID: \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:23.317684 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.317548    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59pbs\" (UniqueName: \"kubernetes.io/projected/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-kube-api-access-59pbs\") pod \"payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r\" (UID: \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:23.317684 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.317601    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77lv4\" (UniqueName: \"kubernetes.io/projected/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-kube-api-access-77lv4\") pod \"payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8\" (UID: \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:23.419265 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.419174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r\" (UID: \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:23.419265 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.419225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8\" (UID: \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:23.419469 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.419252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59pbs\" (UniqueName: \"kubernetes.io/projected/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-kube-api-access-59pbs\") pod \"payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r\" (UID: \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:23.419469 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.419292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77lv4\" (UniqueName: \"kubernetes.io/projected/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-kube-api-access-77lv4\") pod \"payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8\" (UID: \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:23.420015 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.419988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8\" (UID: \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:23.420145 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.420019    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r\" (UID: \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:23.426533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.426503    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77lv4\" (UniqueName: \"kubernetes.io/projected/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-kube-api-access-77lv4\") pod \"payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8\" (UID: \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:23.426533 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.426517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59pbs\" (UniqueName: \"kubernetes.io/projected/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-kube-api-access-59pbs\") pod \"payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r\" (UID: \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\") " pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:23.481734 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.481688    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:23.501044 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.501014    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:23.755344 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.755271    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"]
Sep 10 23:55:23.758784 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:55:23.758763    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e68be5b_5b9b_45d8_87e3_80ad8b3f4de7.slice/crio-59dbd028a2c120151d9fc38e6d3c9709a89bb285f62f000c15edbd6905934dbd WatchSource:0}: Error finding container 59dbd028a2c120151d9fc38e6d3c9709a89bb285f62f000c15edbd6905934dbd: Status 404 returned error can't find the container with id 59dbd028a2c120151d9fc38e6d3c9709a89bb285f62f000c15edbd6905934dbd
Sep 10 23:55:23.784519 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:23.784497    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"]
Sep 10 23:55:23.797105 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:55:23.797075    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8edcbd75_e23d_4829_b6ee_854d72ddbdd4.slice/crio-3713ab6fc5b3ed667a847bd5e33ef1c7e6f30587316046ca26951f92fb701b0a WatchSource:0}: Error finding container 3713ab6fc5b3ed667a847bd5e33ef1c7e6f30587316046ca26951f92fb701b0a: Status 404 returned error can't find the container with id 3713ab6fc5b3ed667a847bd5e33ef1c7e6f30587316046ca26951f92fb701b0a
Sep 10 23:55:24.177555 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:24.177518    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8" event={"ID":"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7","Type":"ContainerStarted","Data":"aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86"}
Sep 10 23:55:24.178011 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:24.177575    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8" event={"ID":"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7","Type":"ContainerStarted","Data":"59dbd028a2c120151d9fc38e6d3c9709a89bb285f62f000c15edbd6905934dbd"}
Sep 10 23:55:24.178011 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:24.177602    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:24.179250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:24.179221    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r" event={"ID":"8edcbd75-e23d-4829-b6ee-854d72ddbdd4","Type":"ContainerStarted","Data":"9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1"}
Sep 10 23:55:24.179364 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:24.179268    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r" event={"ID":"8edcbd75-e23d-4829-b6ee-854d72ddbdd4","Type":"ContainerStarted","Data":"3713ab6fc5b3ed667a847bd5e33ef1c7e6f30587316046ca26951f92fb701b0a"}
Sep 10 23:55:24.179364 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:24.179316    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:24.191567 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:24.191522    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8" podStartSLOduration=1.191504864 podStartE2EDuration="1.191504864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:55:24.190746027 +0000 UTC m=+1337.844466200" watchObservedRunningTime="2026-09-10 23:55:24.191504864 +0000 UTC m=+1337.845225038"
Sep 10 23:55:24.204429 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:24.204387    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r" podStartSLOduration=1.204370225 podStartE2EDuration="1.204370225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:55:24.203731046 +0000 UTC m=+1337.857451218" watchObservedRunningTime="2026-09-10 23:55:24.204370225 +0000 UTC m=+1337.858090400"
Sep 10 23:55:35.187452 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:35.187373    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:55:35.187452 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:35.187446    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:55:52.134392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.134359    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"]
Sep 10 23:55:52.138510 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.138483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:52.141319 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.141283    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-bcad17\""
Sep 10 23:55:52.141471 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.141293    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-bcad17-dockercfg-cqrrv\""
Sep 10 23:55:52.151767 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.151737    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"]
Sep 10 23:55:52.163859 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.163832    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"]
Sep 10 23:55:52.167460 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.167441    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:52.188727 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.188697    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"]
Sep 10 23:55:52.202271 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.202236    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfnck\" (UniqueName: \"kubernetes.io/projected/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-kube-api-access-mfnck\") pod \"payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744\" (UID: \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\") " pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:52.202449 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.202334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ab7cee-4ee8-450c-98d8-62452822b503-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck\" (UID: \"f0ab7cee-4ee8-450c-98d8-62452822b503\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:52.202449 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.202362    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd8mj\" (UniqueName: \"kubernetes.io/projected/f0ab7cee-4ee8-450c-98d8-62452822b503-kube-api-access-nd8mj\") pod \"payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck\" (UID: \"f0ab7cee-4ee8-450c-98d8-62452822b503\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:52.202556 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.202459    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744\" (UID: \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\") " pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:52.303126 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.303087    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ab7cee-4ee8-450c-98d8-62452822b503-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck\" (UID: \"f0ab7cee-4ee8-450c-98d8-62452822b503\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:52.303126 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.303122    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd8mj\" (UniqueName: \"kubernetes.io/projected/f0ab7cee-4ee8-450c-98d8-62452822b503-kube-api-access-nd8mj\") pod \"payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck\" (UID: \"f0ab7cee-4ee8-450c-98d8-62452822b503\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:52.303392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.303170    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744\" (UID: \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\") " pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:52.303392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.303214    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfnck\" (UniqueName: \"kubernetes.io/projected/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-kube-api-access-mfnck\") pod \"payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744\" (UID: \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\") " pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:52.303776 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.303756    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ab7cee-4ee8-450c-98d8-62452822b503-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck\" (UID: \"f0ab7cee-4ee8-450c-98d8-62452822b503\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:52.303986 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.303968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744\" (UID: \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\") " pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:52.311071 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.311042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfnck\" (UniqueName: \"kubernetes.io/projected/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-kube-api-access-mfnck\") pod \"payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744\" (UID: \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\") " pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:52.311218 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.311197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd8mj\" (UniqueName: \"kubernetes.io/projected/f0ab7cee-4ee8-450c-98d8-62452822b503-kube-api-access-nd8mj\") pod \"payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck\" (UID: \"f0ab7cee-4ee8-450c-98d8-62452822b503\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:52.451150 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.451065    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:52.486404 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.486371    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:52.726994 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.726969    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"]
Sep 10 23:55:52.728565 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:55:52.728534    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ab7cee_4ee8_450c_98d8_62452822b503.slice/crio-e06c3a80e2ca3b6468b647d8f826ad2276ab045ea2baa6d3cc52bdcc6bd05a4b WatchSource:0}: Error finding container e06c3a80e2ca3b6468b647d8f826ad2276ab045ea2baa6d3cc52bdcc6bd05a4b: Status 404 returned error can't find the container with id e06c3a80e2ca3b6468b647d8f826ad2276ab045ea2baa6d3cc52bdcc6bd05a4b
Sep 10 23:55:52.771259 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:52.771225    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"]
Sep 10 23:55:52.772226 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:55:52.772194    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a0305c9_e3e0_4dcb_8108_f4c488a4c46c.slice/crio-1d44a1e9ed57d94a76a9c0f1436fd22033e4ab37c31cc0c2c7a5eaf023793e1e WatchSource:0}: Error finding container 1d44a1e9ed57d94a76a9c0f1436fd22033e4ab37c31cc0c2c7a5eaf023793e1e: Status 404 returned error can't find the container with id 1d44a1e9ed57d94a76a9c0f1436fd22033e4ab37c31cc0c2c7a5eaf023793e1e
Sep 10 23:55:53.343569 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:53.343526    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744" event={"ID":"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c","Type":"ContainerStarted","Data":"9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a"}
Sep 10 23:55:53.344039 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:53.343593    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744" event={"ID":"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c","Type":"ContainerStarted","Data":"1d44a1e9ed57d94a76a9c0f1436fd22033e4ab37c31cc0c2c7a5eaf023793e1e"}
Sep 10 23:55:53.344039 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:53.343724    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:55:53.345179 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:53.345153    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck" event={"ID":"f0ab7cee-4ee8-450c-98d8-62452822b503","Type":"ContainerStarted","Data":"8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56"}
Sep 10 23:55:53.345288 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:53.345191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck" event={"ID":"f0ab7cee-4ee8-450c-98d8-62452822b503","Type":"ContainerStarted","Data":"e06c3a80e2ca3b6468b647d8f826ad2276ab045ea2baa6d3cc52bdcc6bd05a4b"}
Sep 10 23:55:53.345288 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:53.345211    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:55:53.358892 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:53.358835    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744" podStartSLOduration=1.358816762 podStartE2EDuration="1.358816762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:55:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:55:53.358365266 +0000 UTC m=+1367.012085440" watchObservedRunningTime="2026-09-10 23:55:53.358816762 +0000 UTC m=+1367.012536936"
Sep 10 23:55:53.373789 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:55:53.373733    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck" podStartSLOduration=1.373714037 podStartE2EDuration="1.373714037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:55:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:55:53.371576369 +0000 UTC m=+1367.025296545" watchObservedRunningTime="2026-09-10 23:55:53.373714037 +0000 UTC m=+1367.027434215"
Sep 10 23:56:04.354830 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:04.354798    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:56:04.355231 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:04.354870    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:56:07.210687 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:07.210656    2569 scope.go:117] "RemoveContainer" containerID="d68e0ef55801dfcf1f351caab6ab9f8284240a84ab96e3f2f9cf4b6ad02f52f7"
Sep 10 23:56:07.221732 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:07.221706    2569 scope.go:117] "RemoveContainer" containerID="23d4ca2d688a90f8af2916c1d7472727b195810bb3cef1fb5c49fea297046607"
Sep 10 23:56:33.655787 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:33.655747    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"]
Sep 10 23:56:33.656501 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:33.656447    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r" podUID="8edcbd75-e23d-4829-b6ee-854d72ddbdd4" containerName="payload-processing" containerID="cri-o://9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1" gracePeriod=30
Sep 10 23:56:33.660677 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:33.660652    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"]
Sep 10 23:56:33.660909 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:33.660867    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8" podUID="6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7" containerName="payload-pre-processing" containerID="cri-o://aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86" gracePeriod=30
Sep 10 23:56:33.935678 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:33.935644    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:56:33.955895 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:33.955871    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:56:34.011380 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.009033    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77lv4\" (UniqueName: \"kubernetes.io/projected/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-kube-api-access-77lv4\") pod \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\" (UID: \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\") "
Sep 10 23:56:34.011380 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.009117    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59pbs\" (UniqueName: \"kubernetes.io/projected/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-kube-api-access-59pbs\") pod \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\" (UID: \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\") "
Sep 10 23:56:34.011380 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.009173    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-plugins-config-volume\") pod \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\" (UID: \"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7\") "
Sep 10 23:56:34.011380 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.009275    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-plugins-config-volume\") pod \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\" (UID: \"8edcbd75-e23d-4829-b6ee-854d72ddbdd4\") "
Sep 10 23:56:34.011380 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.010846    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7" (UID: "6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:56:34.011380 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.011110    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8edcbd75-e23d-4829-b6ee-854d72ddbdd4" (UID: "8edcbd75-e23d-4829-b6ee-854d72ddbdd4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:56:34.014668 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.014607    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-kube-api-access-59pbs" (OuterVolumeSpecName: "kube-api-access-59pbs") pod "8edcbd75-e23d-4829-b6ee-854d72ddbdd4" (UID: "8edcbd75-e23d-4829-b6ee-854d72ddbdd4"). InnerVolumeSpecName "kube-api-access-59pbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:56:34.015293 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.015271    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-kube-api-access-77lv4" (OuterVolumeSpecName: "kube-api-access-77lv4") pod "6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7" (UID: "6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7"). InnerVolumeSpecName "kube-api-access-77lv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:56:34.111021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.110984    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77lv4\" (UniqueName: \"kubernetes.io/projected/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-kube-api-access-77lv4\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:34.111021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.111014    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59pbs\" (UniqueName: \"kubernetes.io/projected/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-kube-api-access-59pbs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:34.111021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.111024    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:34.111021 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.111033    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8edcbd75-e23d-4829-b6ee-854d72ddbdd4-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:34.598268 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.598229    2569 generic.go:356] "Generic (PLEG): container finished" podID="8edcbd75-e23d-4829-b6ee-854d72ddbdd4" containerID="9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1" exitCode=0
Sep 10 23:56:34.598462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.598276    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"
Sep 10 23:56:34.598462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.598319    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r" event={"ID":"8edcbd75-e23d-4829-b6ee-854d72ddbdd4","Type":"ContainerDied","Data":"9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1"}
Sep 10 23:56:34.598462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.598370    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r" event={"ID":"8edcbd75-e23d-4829-b6ee-854d72ddbdd4","Type":"ContainerDied","Data":"3713ab6fc5b3ed667a847bd5e33ef1c7e6f30587316046ca26951f92fb701b0a"}
Sep 10 23:56:34.598462 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.598394    2569 scope.go:117] "RemoveContainer" containerID="9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1"
Sep 10 23:56:34.599845 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.599817    2569 generic.go:356] "Generic (PLEG): container finished" podID="6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7" containerID="aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86" exitCode=0
Sep 10 23:56:34.599995 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.599867    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"
Sep 10 23:56:34.599995 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.599873    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8" event={"ID":"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7","Type":"ContainerDied","Data":"aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86"}
Sep 10 23:56:34.599995 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.599911    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8" event={"ID":"6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7","Type":"ContainerDied","Data":"59dbd028a2c120151d9fc38e6d3c9709a89bb285f62f000c15edbd6905934dbd"}
Sep 10 23:56:34.608476 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.608458    2569 scope.go:117] "RemoveContainer" containerID="9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1"
Sep 10 23:56:34.608745 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:56:34.608726    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1\": container with ID starting with 9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1 not found: ID does not exist" containerID="9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1"
Sep 10 23:56:34.608803 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.608753    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1"} err="failed to get container status \"9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1\": rpc error: code = NotFound desc = could not find container \"9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1\": container with ID starting with 9b9d5402a089e1e87ece2541ac392fcea6cdd105160ee92e542022cf6e7d0ad1 not found: ID does not exist"
Sep 10 23:56:34.608803 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.608768    2569 scope.go:117] "RemoveContainer" containerID="aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86"
Sep 10 23:56:34.616905 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.616881    2569 scope.go:117] "RemoveContainer" containerID="aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86"
Sep 10 23:56:34.617194 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:56:34.617173    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86\": container with ID starting with aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86 not found: ID does not exist" containerID="aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86"
Sep 10 23:56:34.617283 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.617198    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86"} err="failed to get container status \"aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86\": rpc error: code = NotFound desc = could not find container \"aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86\": container with ID starting with aaebb6489a33f343f21ddf95d672516338b47430ba631555df4a380374466e86 not found: ID does not exist"
Sep 10 23:56:34.645227 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.645198    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"]
Sep 10 23:56:34.654715 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.654688    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7c0647-664666cbc7-7kq6r"]
Sep 10 23:56:34.676109 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.676078    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"]
Sep 10 23:56:34.686580 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.686553    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-7c0647-54454bd546-gp4w8"]
Sep 10 23:56:34.979451 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.979418    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7" path="/var/lib/kubelet/pods/6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7/volumes"
Sep 10 23:56:34.979797 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:34.979783    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8edcbd75-e23d-4829-b6ee-854d72ddbdd4" path="/var/lib/kubelet/pods/8edcbd75-e23d-4829-b6ee-854d72ddbdd4/volumes"
Sep 10 23:56:40.696259 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.696225    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"]
Sep 10 23:56:40.696721 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.696644    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8edcbd75-e23d-4829-b6ee-854d72ddbdd4" containerName="payload-processing"
Sep 10 23:56:40.696721 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.696655    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8edcbd75-e23d-4829-b6ee-854d72ddbdd4" containerName="payload-processing"
Sep 10 23:56:40.696721 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.696681    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7" containerName="payload-pre-processing"
Sep 10 23:56:40.696721 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.696687    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7" containerName="payload-pre-processing"
Sep 10 23:56:40.696894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.696757    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8edcbd75-e23d-4829-b6ee-854d72ddbdd4" containerName="payload-processing"
Sep 10 23:56:40.696894 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.696770    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e68be5b-5b9b-45d8-87e3-80ad8b3f4de7" containerName="payload-pre-processing"
Sep 10 23:56:40.699897 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.699881    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"
Sep 10 23:56:40.701807 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.701770    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:56:40.701906 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.701860    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:56:40.706940 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.706915    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"]
Sep 10 23:56:40.772793 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.772756    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh\" (UID: \"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"
Sep 10 23:56:40.873486 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.873451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh\" (UID: \"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"
Sep 10 23:56:40.874149 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:40.874126    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh\" (UID: \"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"
Sep 10 23:56:41.012775 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:41.012687    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"
Sep 10 23:56:41.282209 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:41.282177    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"]
Sep 10 23:56:41.284679 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:56:41.284654    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff16cc4d_65c8_4c0d_b299_e2e0d2fb7c6d.slice/crio-b3d24f6eee0154ae9f67c89ca000405c8209de35269cfeb14d084233ec309e9e WatchSource:0}: Error finding container b3d24f6eee0154ae9f67c89ca000405c8209de35269cfeb14d084233ec309e9e: Status 404 returned error can't find the container with id b3d24f6eee0154ae9f67c89ca000405c8209de35269cfeb14d084233ec309e9e
Sep 10 23:56:41.645918 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:41.645882    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh" event={"ID":"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d","Type":"ContainerStarted","Data":"0305e9887649663a6619ff0789a118015e6af8ee1d03a79c2b21bdbf26802a25"}
Sep 10 23:56:41.646112 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:41.645959    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh" event={"ID":"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d","Type":"ContainerStarted","Data":"b3d24f6eee0154ae9f67c89ca000405c8209de35269cfeb14d084233ec309e9e"}
Sep 10 23:56:41.665497 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:41.665444    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh" podStartSLOduration=1.665427443 podStartE2EDuration="1.665427443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:56:41.662890326 +0000 UTC m=+1415.316610490" watchObservedRunningTime="2026-09-10 23:56:41.665427443 +0000 UTC m=+1415.319147616"
Sep 10 23:56:42.652739 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:42.652703    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d" containerID="0305e9887649663a6619ff0789a118015e6af8ee1d03a79c2b21bdbf26802a25" exitCode=0
Sep 10 23:56:42.653155 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:42.652789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh" event={"ID":"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d","Type":"ContainerDied","Data":"0305e9887649663a6619ff0789a118015e6af8ee1d03a79c2b21bdbf26802a25"}
Sep 10 23:56:43.798129 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:43.798102    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"
Sep 10 23:56:43.904463 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:43.904423    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d-maas-api-service-ca\") pod \"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d\" (UID: \"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d\") "
Sep 10 23:56:43.904812 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:43.904790    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d" (UID: "ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:56:44.005653 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:44.005563    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:44.665357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:44.665322    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh" event={"ID":"ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d","Type":"ContainerDied","Data":"b3d24f6eee0154ae9f67c89ca000405c8209de35269cfeb14d084233ec309e9e"}
Sep 10 23:56:44.665535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:44.665374    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3d24f6eee0154ae9f67c89ca000405c8209de35269cfeb14d084233ec309e9e"
Sep 10 23:56:44.665535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:44.665337    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"
Sep 10 23:56:56.269287 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.269245    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"]
Sep 10 23:56:56.269763 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.269488    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744" podUID="4a0305c9-e3e0-4dcb-8108-f4c488a4c46c" containerName="payload-processing" containerID="cri-o://9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a" gracePeriod=30
Sep 10 23:56:56.281198 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.281163    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"]
Sep 10 23:56:56.281414 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.281385    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck" podUID="f0ab7cee-4ee8-450c-98d8-62452822b503" containerName="payload-pre-processing" containerID="cri-o://8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56" gracePeriod=30
Sep 10 23:56:56.453850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.453786    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"]
Sep 10 23:56:56.456087 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.456059    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-bcad17-8cjnh"]
Sep 10 23:56:56.616609 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.616561    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:56:56.620094 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.620073    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:56:56.732198 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.732165    2569 generic.go:356] "Generic (PLEG): container finished" podID="4a0305c9-e3e0-4dcb-8108-f4c488a4c46c" containerID="9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a" exitCode=0
Sep 10 23:56:56.732370 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.732215    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"
Sep 10 23:56:56.732370 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.732250    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744" event={"ID":"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c","Type":"ContainerDied","Data":"9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a"}
Sep 10 23:56:56.732370 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.732303    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744" event={"ID":"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c","Type":"ContainerDied","Data":"1d44a1e9ed57d94a76a9c0f1436fd22033e4ab37c31cc0c2c7a5eaf023793e1e"}
Sep 10 23:56:56.732370 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.732331    2569 scope.go:117] "RemoveContainer" containerID="9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a"
Sep 10 23:56:56.733631 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.733608    2569 generic.go:356] "Generic (PLEG): container finished" podID="f0ab7cee-4ee8-450c-98d8-62452822b503" containerID="8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56" exitCode=0
Sep 10 23:56:56.733736 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.733650    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"
Sep 10 23:56:56.733736 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.733678    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck" event={"ID":"f0ab7cee-4ee8-450c-98d8-62452822b503","Type":"ContainerDied","Data":"8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56"}
Sep 10 23:56:56.733736 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.733713    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck" event={"ID":"f0ab7cee-4ee8-450c-98d8-62452822b503","Type":"ContainerDied","Data":"e06c3a80e2ca3b6468b647d8f826ad2276ab045ea2baa6d3cc52bdcc6bd05a4b"}
Sep 10 23:56:56.742062 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.742034    2569 scope.go:117] "RemoveContainer" containerID="9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a"
Sep 10 23:56:56.742325 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:56:56.742307    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a\": container with ID starting with 9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a not found: ID does not exist" containerID="9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a"
Sep 10 23:56:56.742390 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.742334    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a"} err="failed to get container status \"9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a\": rpc error: code = NotFound desc = could not find container \"9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a\": container with ID starting with 9e2602c7bb2c7e1027a7d4f3323b5b6c7132fa9edc6cc565d8974fa8f632801a not found: ID does not exist"
Sep 10 23:56:56.742390 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.742350    2569 scope.go:117] "RemoveContainer" containerID="8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56"
Sep 10 23:56:56.750302 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.750284    2569 scope.go:117] "RemoveContainer" containerID="8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56"
Sep 10 23:56:56.750582 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:56:56.750560    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56\": container with ID starting with 8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56 not found: ID does not exist" containerID="8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56"
Sep 10 23:56:56.750640 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.750589    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56"} err="failed to get container status \"8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56\": rpc error: code = NotFound desc = could not find container \"8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56\": container with ID starting with 8cdeadeaffaf7a7388547e954e4e24be3c5b6d5a1d55eaf12182a1ae358cfc56 not found: ID does not exist"
Sep 10 23:56:56.754014 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.753996    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ab7cee-4ee8-450c-98d8-62452822b503-plugins-config-volume\") pod \"f0ab7cee-4ee8-450c-98d8-62452822b503\" (UID: \"f0ab7cee-4ee8-450c-98d8-62452822b503\") "
Sep 10 23:56:56.754111 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.754037    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-plugins-config-volume\") pod \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\" (UID: \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\") "
Sep 10 23:56:56.754111 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.754076    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd8mj\" (UniqueName: \"kubernetes.io/projected/f0ab7cee-4ee8-450c-98d8-62452822b503-kube-api-access-nd8mj\") pod \"f0ab7cee-4ee8-450c-98d8-62452822b503\" (UID: \"f0ab7cee-4ee8-450c-98d8-62452822b503\") "
Sep 10 23:56:56.754328 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.754118    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfnck\" (UniqueName: \"kubernetes.io/projected/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-kube-api-access-mfnck\") pod \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\" (UID: \"4a0305c9-e3e0-4dcb-8108-f4c488a4c46c\") "
Sep 10 23:56:56.754439 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.754417    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4a0305c9-e3e0-4dcb-8108-f4c488a4c46c" (UID: "4a0305c9-e3e0-4dcb-8108-f4c488a4c46c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:56:56.754514 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.754446    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0ab7cee-4ee8-450c-98d8-62452822b503-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0ab7cee-4ee8-450c-98d8-62452822b503" (UID: "f0ab7cee-4ee8-450c-98d8-62452822b503"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:56:56.754514 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.754505    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0ab7cee-4ee8-450c-98d8-62452822b503-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:56.754618 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.754524    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:56.756227 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.756205    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-kube-api-access-mfnck" (OuterVolumeSpecName: "kube-api-access-mfnck") pod "4a0305c9-e3e0-4dcb-8108-f4c488a4c46c" (UID: "4a0305c9-e3e0-4dcb-8108-f4c488a4c46c"). InnerVolumeSpecName "kube-api-access-mfnck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:56:56.756227 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.756222    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0ab7cee-4ee8-450c-98d8-62452822b503-kube-api-access-nd8mj" (OuterVolumeSpecName: "kube-api-access-nd8mj") pod "f0ab7cee-4ee8-450c-98d8-62452822b503" (UID: "f0ab7cee-4ee8-450c-98d8-62452822b503"). InnerVolumeSpecName "kube-api-access-nd8mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:56:56.855348 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.855265    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd8mj\" (UniqueName: \"kubernetes.io/projected/f0ab7cee-4ee8-450c-98d8-62452822b503-kube-api-access-nd8mj\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:56.855348 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.855299    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfnck\" (UniqueName: \"kubernetes.io/projected/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c-kube-api-access-mfnck\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:56:56.980346 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:56.980311    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d" path="/var/lib/kubelet/pods/ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d/volumes"
Sep 10 23:56:57.045644 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:57.045608    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"]
Sep 10 23:56:57.049563 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:57.049538    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-bcad17-6dd87bfcb5-xb744"]
Sep 10 23:56:57.057254 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:57.057227    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"]
Sep 10 23:56:57.060099 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:57.060074    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-bcad17-69bfb7b7d-986ck"]
Sep 10 23:56:58.979217 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:58.979185    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a0305c9-e3e0-4dcb-8108-f4c488a4c46c" path="/var/lib/kubelet/pods/4a0305c9-e3e0-4dcb-8108-f4c488a4c46c/volumes"
Sep 10 23:56:58.979652 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:56:58.979552    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0ab7cee-4ee8-450c-98d8-62452822b503" path="/var/lib/kubelet/pods/f0ab7cee-4ee8-450c-98d8-62452822b503/volumes"
Sep 10 23:57:03.077302 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077266    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"]
Sep 10 23:57:03.077717 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077702    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a0305c9-e3e0-4dcb-8108-f4c488a4c46c" containerName="payload-processing"
Sep 10 23:57:03.077761 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077719    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a0305c9-e3e0-4dcb-8108-f4c488a4c46c" containerName="payload-processing"
Sep 10 23:57:03.077761 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077734    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d" containerName="revoke-keys"
Sep 10 23:57:03.077761 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077739    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d" containerName="revoke-keys"
Sep 10 23:57:03.077861 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077763    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0ab7cee-4ee8-450c-98d8-62452822b503" containerName="payload-pre-processing"
Sep 10 23:57:03.077861 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077769    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0ab7cee-4ee8-450c-98d8-62452822b503" containerName="payload-pre-processing"
Sep 10 23:57:03.077861 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077836    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a0305c9-e3e0-4dcb-8108-f4c488a4c46c" containerName="payload-processing"
Sep 10 23:57:03.077861 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077846    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0ab7cee-4ee8-450c-98d8-62452822b503" containerName="payload-pre-processing"
Sep 10 23:57:03.077861 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.077852    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff16cc4d-65c8-4c0d-b299-e2e0d2fb7c6d" containerName="revoke-keys"
Sep 10 23:57:03.082332 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.082314    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"
Sep 10 23:57:03.085314 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.085291    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:57:03.085655 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.085367    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:57:03.087223 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.087198    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"]
Sep 10 23:57:03.109178 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.109142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b9186e-ea8a-42c2-836d-31df33519c0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx\" (UID: \"a9b9186e-ea8a-42c2-836d-31df33519c0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"
Sep 10 23:57:03.210774 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.210724    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b9186e-ea8a-42c2-836d-31df33519c0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx\" (UID: \"a9b9186e-ea8a-42c2-836d-31df33519c0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"
Sep 10 23:57:03.211384 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.211362    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b9186e-ea8a-42c2-836d-31df33519c0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx\" (UID: \"a9b9186e-ea8a-42c2-836d-31df33519c0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"
Sep 10 23:57:03.396332 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.396304    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"
Sep 10 23:57:03.666187 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.666154    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"]
Sep 10 23:57:03.669237 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:57:03.669205    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b9186e_ea8a_42c2_836d_31df33519c0b.slice/crio-b932420f83b41714df35938cec14516ab2afe702e83c0cd972b7507e4142163c WatchSource:0}: Error finding container b932420f83b41714df35938cec14516ab2afe702e83c0cd972b7507e4142163c: Status 404 returned error can't find the container with id b932420f83b41714df35938cec14516ab2afe702e83c0cd972b7507e4142163c
Sep 10 23:57:03.775785 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:03.775751    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx" event={"ID":"a9b9186e-ea8a-42c2-836d-31df33519c0b","Type":"ContainerStarted","Data":"b932420f83b41714df35938cec14516ab2afe702e83c0cd972b7507e4142163c"}
Sep 10 23:57:04.783296 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:04.783261    2569 generic.go:356] "Generic (PLEG): container finished" podID="a9b9186e-ea8a-42c2-836d-31df33519c0b" containerID="edccef9406b6476596646dcd6a4e6742327e21cad205d9f0a4d3d637f7ebb67d" exitCode=0
Sep 10 23:57:04.783674 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:04.783354    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx" event={"ID":"a9b9186e-ea8a-42c2-836d-31df33519c0b","Type":"ContainerDied","Data":"edccef9406b6476596646dcd6a4e6742327e21cad205d9f0a4d3d637f7ebb67d"}
Sep 10 23:57:05.932948 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:05.932915    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"
Sep 10 23:57:06.041233 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:06.041149    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b9186e-ea8a-42c2-836d-31df33519c0b-maas-api-service-ca\") pod \"a9b9186e-ea8a-42c2-836d-31df33519c0b\" (UID: \"a9b9186e-ea8a-42c2-836d-31df33519c0b\") "
Sep 10 23:57:06.041514 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:06.041493    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b9186e-ea8a-42c2-836d-31df33519c0b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a9b9186e-ea8a-42c2-836d-31df33519c0b" (UID: "a9b9186e-ea8a-42c2-836d-31df33519c0b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:57:06.142701 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:06.142659    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9b9186e-ea8a-42c2-836d-31df33519c0b-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:06.795718 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:06.795685    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx" event={"ID":"a9b9186e-ea8a-42c2-836d-31df33519c0b","Type":"ContainerDied","Data":"b932420f83b41714df35938cec14516ab2afe702e83c0cd972b7507e4142163c"}
Sep 10 23:57:06.795893 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:06.795736    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b932420f83b41714df35938cec14516ab2afe702e83c0cd972b7507e4142163c"
Sep 10 23:57:06.795893 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:06.795698    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"
Sep 10 23:57:07.247702 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:07.247674    2569 scope.go:117] "RemoveContainer" containerID="357fc7fa47357d769eef844b2a3a9e6c2df29128dc2a0b581b75297d52196e89"
Sep 10 23:57:07.256677 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:07.256652    2569 scope.go:117] "RemoveContainer" containerID="b70c644e4b52a24df39bd47c2217b4f97a302b10d44b2be0d3e00db843ae37ec"
Sep 10 23:57:13.882889 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:13.882852    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"]
Sep 10 23:57:13.883518 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:13.883080    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp" podUID="a14ffa32-7d12-4122-ad2d-c6b1398765b1" containerName="payload-processing" containerID="cri-o://fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b" gracePeriod=30
Sep 10 23:57:13.888392 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:13.888348    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"]
Sep 10 23:57:13.888658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:13.888630    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh" podUID="5e6438f0-ef81-4b4b-94b5-d21f7e3b295f" containerName="payload-pre-processing" containerID="cri-o://5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1" gracePeriod=30
Sep 10 23:57:14.026709 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.026642    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"]
Sep 10 23:57:14.028532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.028504    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4e0933-lb8tx"]
Sep 10 23:57:14.164801 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.164764    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:57:14.221786 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.221761    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:57:14.224173 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.224151    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a14ffa32-7d12-4122-ad2d-c6b1398765b1-plugins-config-volume\") pod \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\" (UID: \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\") "
Sep 10 23:57:14.224243 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.224226    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mhpn\" (UniqueName: \"kubernetes.io/projected/a14ffa32-7d12-4122-ad2d-c6b1398765b1-kube-api-access-2mhpn\") pod \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\" (UID: \"a14ffa32-7d12-4122-ad2d-c6b1398765b1\") "
Sep 10 23:57:14.224464 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.224440    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a14ffa32-7d12-4122-ad2d-c6b1398765b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a14ffa32-7d12-4122-ad2d-c6b1398765b1" (UID: "a14ffa32-7d12-4122-ad2d-c6b1398765b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:57:14.224608 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.224581    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a14ffa32-7d12-4122-ad2d-c6b1398765b1-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:14.226235 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.226214    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a14ffa32-7d12-4122-ad2d-c6b1398765b1-kube-api-access-2mhpn" (OuterVolumeSpecName: "kube-api-access-2mhpn") pod "a14ffa32-7d12-4122-ad2d-c6b1398765b1" (UID: "a14ffa32-7d12-4122-ad2d-c6b1398765b1"). InnerVolumeSpecName "kube-api-access-2mhpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:57:14.324962 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.324907    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nthz8\" (UniqueName: \"kubernetes.io/projected/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-kube-api-access-nthz8\") pod \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\" (UID: \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\") "
Sep 10 23:57:14.325145 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.325046    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-plugins-config-volume\") pod \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\" (UID: \"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f\") "
Sep 10 23:57:14.325368 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.325349    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mhpn\" (UniqueName: \"kubernetes.io/projected/a14ffa32-7d12-4122-ad2d-c6b1398765b1-kube-api-access-2mhpn\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:14.325425 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.325384    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e6438f0-ef81-4b4b-94b5-d21f7e3b295f" (UID: "5e6438f0-ef81-4b4b-94b5-d21f7e3b295f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:57:14.326993 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.326969    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-kube-api-access-nthz8" (OuterVolumeSpecName: "kube-api-access-nthz8") pod "5e6438f0-ef81-4b4b-94b5-d21f7e3b295f" (UID: "5e6438f0-ef81-4b4b-94b5-d21f7e3b295f"). InnerVolumeSpecName "kube-api-access-nthz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:57:14.425903 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.425812    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:14.425903 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.425844    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nthz8\" (UniqueName: \"kubernetes.io/projected/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f-kube-api-access-nthz8\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:14.839916 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.839827    2569 generic.go:356] "Generic (PLEG): container finished" podID="a14ffa32-7d12-4122-ad2d-c6b1398765b1" containerID="fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b" exitCode=0
Sep 10 23:57:14.839916 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.839885    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"
Sep 10 23:57:14.839916 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.839884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp" event={"ID":"a14ffa32-7d12-4122-ad2d-c6b1398765b1","Type":"ContainerDied","Data":"fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b"}
Sep 10 23:57:14.840250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.839977    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp" event={"ID":"a14ffa32-7d12-4122-ad2d-c6b1398765b1","Type":"ContainerDied","Data":"6fd83924a70c01a460b8b3eda1b543e095fcf2e4e72f671b458958579c72d5e3"}
Sep 10 23:57:14.840250 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.840011    2569 scope.go:117] "RemoveContainer" containerID="fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b"
Sep 10 23:57:14.841763 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.841739    2569 generic.go:356] "Generic (PLEG): container finished" podID="5e6438f0-ef81-4b4b-94b5-d21f7e3b295f" containerID="5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1" exitCode=0
Sep 10 23:57:14.841875 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.841799    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"
Sep 10 23:57:14.841965 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.841906    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh" event={"ID":"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f","Type":"ContainerDied","Data":"5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1"}
Sep 10 23:57:14.842089 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.841979    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh" event={"ID":"5e6438f0-ef81-4b4b-94b5-d21f7e3b295f","Type":"ContainerDied","Data":"a81315152664ec1d90ed805594c6c4a079c7286198f039fb8f13902db5f020ca"}
Sep 10 23:57:14.852617 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.852599    2569 scope.go:117] "RemoveContainer" containerID="fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b"
Sep 10 23:57:14.852983 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:57:14.852918    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b\": container with ID starting with fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b not found: ID does not exist" containerID="fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b"
Sep 10 23:57:14.853102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.852993    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b"} err="failed to get container status \"fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b\": rpc error: code = NotFound desc = could not find container \"fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b\": container with ID starting with fdaa8c327b09900aafc10bf7a924e6b58b1abb45fc09fd79d6418874866b466b not found: ID does not exist"
Sep 10 23:57:14.853102 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.853018    2569 scope.go:117] "RemoveContainer" containerID="5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1"
Sep 10 23:57:14.861056 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.861035    2569 scope.go:117] "RemoveContainer" containerID="5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1"
Sep 10 23:57:14.861357 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:57:14.861338    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1\": container with ID starting with 5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1 not found: ID does not exist" containerID="5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1"
Sep 10 23:57:14.861416 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.861364    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1"} err="failed to get container status \"5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1\": rpc error: code = NotFound desc = could not find container \"5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1\": container with ID starting with 5e49f5ea31e5cdc0fb5f5a6e21213418cc4b8fada51535add04bf6269381dfb1 not found: ID does not exist"
Sep 10 23:57:14.865340 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.865308    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"]
Sep 10 23:57:14.867259 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.867237    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-4e0933-56f96d8bdf-7zmtp"]
Sep 10 23:57:14.878585 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.878561    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"]
Sep 10 23:57:14.882701 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.882680    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4e0933-6d95b65f78-nrxbh"]
Sep 10 23:57:14.978690 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.978661    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e6438f0-ef81-4b4b-94b5-d21f7e3b295f" path="/var/lib/kubelet/pods/5e6438f0-ef81-4b4b-94b5-d21f7e3b295f/volumes"
Sep 10 23:57:14.979063 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.979049    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a14ffa32-7d12-4122-ad2d-c6b1398765b1" path="/var/lib/kubelet/pods/a14ffa32-7d12-4122-ad2d-c6b1398765b1/volumes"
Sep 10 23:57:14.979342 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:14.979331    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9b9186e-ea8a-42c2-836d-31df33519c0b" path="/var/lib/kubelet/pods/a9b9186e-ea8a-42c2-836d-31df33519c0b/volumes"
Sep 10 23:57:20.436953 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.436905    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"]
Sep 10 23:57:20.437407 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.437178    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy" containerID="cri-o://2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868" gracePeriod=30
Sep 10 23:57:20.784067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.783976    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"]
Sep 10 23:57:20.784383 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784370    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a14ffa32-7d12-4122-ad2d-c6b1398765b1" containerName="payload-processing"
Sep 10 23:57:20.784432 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784386    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a14ffa32-7d12-4122-ad2d-c6b1398765b1" containerName="payload-processing"
Sep 10 23:57:20.784432 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784404    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e6438f0-ef81-4b4b-94b5-d21f7e3b295f" containerName="payload-pre-processing"
Sep 10 23:57:20.784432 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784409    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e6438f0-ef81-4b4b-94b5-d21f7e3b295f" containerName="payload-pre-processing"
Sep 10 23:57:20.784432 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784419    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9b9186e-ea8a-42c2-836d-31df33519c0b" containerName="revoke-keys"
Sep 10 23:57:20.784432 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784425    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9b9186e-ea8a-42c2-836d-31df33519c0b" containerName="revoke-keys"
Sep 10 23:57:20.784590 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784489    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a14ffa32-7d12-4122-ad2d-c6b1398765b1" containerName="payload-processing"
Sep 10 23:57:20.784590 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784498    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9b9186e-ea8a-42c2-836d-31df33519c0b" containerName="revoke-keys"
Sep 10 23:57:20.784590 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.784506    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e6438f0-ef81-4b4b-94b5-d21f7e3b295f" containerName="payload-pre-processing"
Sep 10 23:57:20.788917 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.788898    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"
Sep 10 23:57:20.791183 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.790967    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 23:57:20.791335 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.791210    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 10 23:57:20.794131 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.794105    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"]
Sep 10 23:57:20.887544 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.887515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68cb3b3f-e6de-4bb2-bc0d-2f2099073f02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx\" (UID: \"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"
Sep 10 23:57:20.988978 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.988926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68cb3b3f-e6de-4bb2-bc0d-2f2099073f02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx\" (UID: \"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"
Sep 10 23:57:20.989847 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:20.989815    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68cb3b3f-e6de-4bb2-bc0d-2f2099073f02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx\" (UID: \"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"
Sep 10 23:57:21.101984 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:21.101880    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"
Sep 10 23:57:21.372808 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:21.372232    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"]
Sep 10 23:57:21.379134 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:57:21.379092    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68cb3b3f_e6de_4bb2_bc0d_2f2099073f02.slice/crio-ce5bd6fe10afa2bfe3358084b16de5a65ad84b5ca507ca0f029721981f610cbc WatchSource:0}: Error finding container ce5bd6fe10afa2bfe3358084b16de5a65ad84b5ca507ca0f029721981f610cbc: Status 404 returned error can't find the container with id ce5bd6fe10afa2bfe3358084b16de5a65ad84b5ca507ca0f029721981f610cbc
Sep 10 23:57:21.881227 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:21.881191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx" event={"ID":"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02","Type":"ContainerStarted","Data":"56b03edfbdc9e19972a73af4f5261c7a059fcc46cbf9c70e5bd87147e32287bb"}
Sep 10 23:57:21.881620 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:21.881243    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx" event={"ID":"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02","Type":"ContainerStarted","Data":"ce5bd6fe10afa2bfe3358084b16de5a65ad84b5ca507ca0f029721981f610cbc"}
Sep 10 23:57:21.894992 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:21.894929    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx" podStartSLOduration=1.8949150750000001 podStartE2EDuration="1.894915075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:57:21.893373184 +0000 UTC m=+1455.547093359" watchObservedRunningTime="2026-09-10 23:57:21.894915075 +0000 UTC m=+1455.548635247"
Sep 10 23:57:22.065923 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:22.065891    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 23:57:22.066067 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:22.065973    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 23:57:22.887581 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:22.887543    2569 generic.go:356] "Generic (PLEG): container finished" podID="68cb3b3f-e6de-4bb2-bc0d-2f2099073f02" containerID="56b03edfbdc9e19972a73af4f5261c7a059fcc46cbf9c70e5bd87147e32287bb" exitCode=0
Sep 10 23:57:22.887990 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:22.887618    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx" event={"ID":"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02","Type":"ContainerDied","Data":"56b03edfbdc9e19972a73af4f5261c7a059fcc46cbf9c70e5bd87147e32287bb"}
Sep 10 23:57:24.026718 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:24.026690    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"
Sep 10 23:57:24.120099 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:24.120046    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68cb3b3f-e6de-4bb2-bc0d-2f2099073f02-maas-api-service-ca\") pod \"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02\" (UID: \"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02\") "
Sep 10 23:57:24.120480 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:24.120457    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68cb3b3f-e6de-4bb2-bc0d-2f2099073f02-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "68cb3b3f-e6de-4bb2-bc0d-2f2099073f02" (UID: "68cb3b3f-e6de-4bb2-bc0d-2f2099073f02"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:57:24.120558 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:24.120543    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68cb3b3f-e6de-4bb2-bc0d-2f2099073f02-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:24.899532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:24.899505    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"
Sep 10 23:57:24.899710 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:24.899532    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx" event={"ID":"68cb3b3f-e6de-4bb2-bc0d-2f2099073f02","Type":"ContainerDied","Data":"ce5bd6fe10afa2bfe3358084b16de5a65ad84b5ca507ca0f029721981f610cbc"}
Sep 10 23:57:24.899710 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:24.899576    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce5bd6fe10afa2bfe3358084b16de5a65ad84b5ca507ca0f029721981f610cbc"
Sep 10 23:57:25.699282 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.699219    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:57:25.837880 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.837847    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-token\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.837880 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.837883    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-podinfo\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.838121 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.837912    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-certs\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.838121 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838056    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55qrn\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-kube-api-access-55qrn\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.838213 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838125    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-socket\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.838213 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838184    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-envoy\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.838325 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838224    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istiod-ca-cert\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.838325 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838252    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-credential-socket\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.838325 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838262    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:57:25.838325 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838276    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-data\") pod \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\" (UID: \"c0ae0aea-b104-43e7-8435-f9087ec99fb4\") "
Sep 10 23:57:25.838850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838560    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:57:25.838850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838593    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:57:25.838850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838677    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:25.838850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838696    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:25.838850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838790    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:57:25.838850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838811    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:25.838850 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.838818    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-data" (OuterVolumeSpecName: "istio-data") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:57:25.840362 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.840332    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 23:57:25.840489 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.840385    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-kube-api-access-55qrn" (OuterVolumeSpecName: "kube-api-access-55qrn") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "kube-api-access-55qrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:57:25.840556 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.840507    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-token" (OuterVolumeSpecName: "istio-token") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:57:25.840556 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.840513    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c0ae0aea-b104-43e7-8435-f9087ec99fb4" (UID: "c0ae0aea-b104-43e7-8435-f9087ec99fb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 23:57:25.913185 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.913142    2569 generic.go:356] "Generic (PLEG): container finished" podID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerID="2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868" exitCode=0
Sep 10 23:57:25.913388 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.913204    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"
Sep 10 23:57:25.913388 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.913223    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" event={"ID":"c0ae0aea-b104-43e7-8435-f9087ec99fb4","Type":"ContainerDied","Data":"2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868"}
Sep 10 23:57:25.913388 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.913320    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb" event={"ID":"c0ae0aea-b104-43e7-8435-f9087ec99fb4","Type":"ContainerDied","Data":"54d0699b2d47c6b21f91649905d46fcfd6b1c5d9afec6b5e512da7ac4507432d"}
Sep 10 23:57:25.913388 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.913352    2569 scope.go:117] "RemoveContainer" containerID="2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868"
Sep 10 23:57:25.922752 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.922735    2569 scope.go:117] "RemoveContainer" containerID="12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99"
Sep 10 23:57:25.930970 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.930947    2569 scope.go:117] "RemoveContainer" containerID="2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868"
Sep 10 23:57:25.931214 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:57:25.931198    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868\": container with ID starting with 2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868 not found: ID does not exist" containerID="2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868"
Sep 10 23:57:25.931265 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.931234    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868"} err="failed to get container status \"2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868\": rpc error: code = NotFound desc = could not find container \"2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868\": container with ID starting with 2f00b9d71717ba5b9ecc3287adf3236cff994eb0f2378bfa2e8853c616b5f868 not found: ID does not exist"
Sep 10 23:57:25.931265 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.931251    2569 scope.go:117] "RemoveContainer" containerID="12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99"
Sep 10 23:57:25.931493 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:57:25.931479    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99\": container with ID starting with 12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99 not found: ID does not exist" containerID="12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99"
Sep 10 23:57:25.931532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.931496    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99"} err="failed to get container status \"12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99\": rpc error: code = NotFound desc = could not find container \"12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99\": container with ID starting with 12e0c04a656d70092559f0af54f5e79a111928bc7a8b44ff1e22cd72beffae99 not found: ID does not exist"
Sep 10 23:57:25.934805 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.934779    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"]
Sep 10 23:57:25.938317 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.938295    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4e0933-openshift-default-cbf675654-qkqrb"]
Sep 10 23:57:25.939574 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.939556    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55qrn\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-kube-api-access-55qrn\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:25.939628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.939577    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:25.939628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.939587    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:25.939628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.939595    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:25.939628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.939603    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:25.939628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:25.939610    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0ae0aea-b104-43e7-8435-f9087ec99fb4-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:26.978025 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:26.977987    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" path="/var/lib/kubelet/pods/c0ae0aea-b104-43e7-8435-f9087ec99fb4/volumes"
Sep 10 23:57:31.225115 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.225081    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"]
Sep 10 23:57:31.225663 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.225632    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9" podUID="c72380a7-362a-4582-a4e9-11d4d8189ad9" containerName="payload-pre-processing" containerID="cri-o://0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8" gracePeriod=30
Sep 10 23:57:31.370242 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.370200    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"]
Sep 10 23:57:31.373036 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.373005    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-57be92-cbxdx"]
Sep 10 23:57:31.492119 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.492053    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:57:31.492988 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.492963    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c72380a7-362a-4582-a4e9-11d4d8189ad9-plugins-config-volume\") pod \"c72380a7-362a-4582-a4e9-11d4d8189ad9\" (UID: \"c72380a7-362a-4582-a4e9-11d4d8189ad9\") "
Sep 10 23:57:31.493105 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.493032    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v65qt\" (UniqueName: \"kubernetes.io/projected/c72380a7-362a-4582-a4e9-11d4d8189ad9-kube-api-access-v65qt\") pod \"c72380a7-362a-4582-a4e9-11d4d8189ad9\" (UID: \"c72380a7-362a-4582-a4e9-11d4d8189ad9\") "
Sep 10 23:57:31.493298 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.493275    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c72380a7-362a-4582-a4e9-11d4d8189ad9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c72380a7-362a-4582-a4e9-11d4d8189ad9" (UID: "c72380a7-362a-4582-a4e9-11d4d8189ad9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 23:57:31.495028 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.495004    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c72380a7-362a-4582-a4e9-11d4d8189ad9-kube-api-access-v65qt" (OuterVolumeSpecName: "kube-api-access-v65qt") pod "c72380a7-362a-4582-a4e9-11d4d8189ad9" (UID: "c72380a7-362a-4582-a4e9-11d4d8189ad9"). InnerVolumeSpecName "kube-api-access-v65qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 23:57:31.593765 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.593728    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v65qt\" (UniqueName: \"kubernetes.io/projected/c72380a7-362a-4582-a4e9-11d4d8189ad9-kube-api-access-v65qt\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:31.593765 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.593760    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c72380a7-362a-4582-a4e9-11d4d8189ad9-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 10 23:57:31.948064 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.948032    2569 generic.go:356] "Generic (PLEG): container finished" podID="c72380a7-362a-4582-a4e9-11d4d8189ad9" containerID="0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8" exitCode=0
Sep 10 23:57:31.948244 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.948076    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"
Sep 10 23:57:31.948244 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.948117    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9" event={"ID":"c72380a7-362a-4582-a4e9-11d4d8189ad9","Type":"ContainerDied","Data":"0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8"}
Sep 10 23:57:31.948244 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.948166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9" event={"ID":"c72380a7-362a-4582-a4e9-11d4d8189ad9","Type":"ContainerDied","Data":"d605c2810d590f08ab0b4150375173152382fd62df97ebdc88a7b8bf192925db"}
Sep 10 23:57:31.948244 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.948189    2569 scope.go:117] "RemoveContainer" containerID="0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8"
Sep 10 23:57:31.957502 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.957485    2569 scope.go:117] "RemoveContainer" containerID="0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8"
Sep 10 23:57:31.957828 ip-10-0-143-154 kubenswrapper[2569]: E0910 23:57:31.957808    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8\": container with ID starting with 0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8 not found: ID does not exist" containerID="0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8"
Sep 10 23:57:31.957893 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.957834    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8"} err="failed to get container status \"0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8\": rpc error: code = NotFound desc = could not find container \"0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8\": container with ID starting with 0e01b4a5527d47cd30d30ba0e5bfa3772bb9fdde0170d3932fd2d3d067fd0cf8 not found: ID does not exist"
Sep 10 23:57:31.967588 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.967564    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"]
Sep 10 23:57:31.971313 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:31.971291    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-57be92-6b6c955bbb-4bbp9"]
Sep 10 23:57:32.978625 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:32.978590    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68cb3b3f-e6de-4bb2-bc0d-2f2099073f02" path="/var/lib/kubelet/pods/68cb3b3f-e6de-4bb2-bc0d-2f2099073f02/volumes"
Sep 10 23:57:32.979043 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:57:32.978988    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c72380a7-362a-4582-a4e9-11d4d8189ad9" path="/var/lib/kubelet/pods/c72380a7-362a-4582-a4e9-11d4d8189ad9/volumes"
Sep 10 23:58:07.102322 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:07.102297    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:58:07.102817 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:07.102297    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 10 23:58:07.108122 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:07.108100    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:58:07.108238 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:07.108103    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 10 23:58:12.662348 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.662311    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"]
Sep 10 23:58:12.665160 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665133    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy"
Sep 10 23:58:12.665160 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665161    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy"
Sep 10 23:58:12.665357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665245    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cb3b3f-e6de-4bb2-bc0d-2f2099073f02" containerName="revoke-keys"
Sep 10 23:58:12.665357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665254    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cb3b3f-e6de-4bb2-bc0d-2f2099073f02" containerName="revoke-keys"
Sep 10 23:58:12.665357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665273    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c72380a7-362a-4582-a4e9-11d4d8189ad9" containerName="payload-pre-processing"
Sep 10 23:58:12.665357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665282    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c72380a7-362a-4582-a4e9-11d4d8189ad9" containerName="payload-pre-processing"
Sep 10 23:58:12.665554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665459    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="68cb3b3f-e6de-4bb2-bc0d-2f2099073f02" containerName="revoke-keys"
Sep 10 23:58:12.665554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665478    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy"
Sep 10 23:58:12.665554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665494    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c72380a7-362a-4582-a4e9-11d4d8189ad9" containerName="payload-pre-processing"
Sep 10 23:58:12.665554 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.665504    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy"
Sep 10 23:58:12.671535 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.671513    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.674418 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.674388    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-71c1cb-openshift-default-dockercfg-kq5l7\""
Sep 10 23:58:12.676168 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.676141    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"]
Sep 10 23:58:12.681778 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.681741    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-envoy\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.681899 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.681802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-podinfo\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.681899 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.681846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-data\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.682094 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.681898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istiod-ca-cert\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.682834 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.682798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrz6x\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-kube-api-access-lrz6x\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.682964 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.682890    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-certs\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.683042 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.682929    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-token\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.683042 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.683006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-socket\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.683165 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.683146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-credential-socket\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.784639 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.784598    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istiod-ca-cert\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.784854 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.784650    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrz6x\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-kube-api-access-lrz6x\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.784854 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.784683    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-certs\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.784854 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.784707    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-token\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.784854 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.784736    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-socket\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.784854 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.784799    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-credential-socket\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.785187 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.784912    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-envoy\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.785187 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.784967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-podinfo\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.785187 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.785006    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-data\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.785352 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.785256    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-certs\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.785515 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.785488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-data\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.785582 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.785556    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-credential-socket\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.785658 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.785638    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istiod-ca-cert\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.786026 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.786003    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-socket\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.788267 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.788238    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-envoy\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.788466 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.788438    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-podinfo\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.793011 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.792988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-token\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.793194 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.793175    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrz6x\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-kube-api-access-lrz6x\") pod \"e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") " pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:12.986709 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:12.986621    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:13.263127 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:13.262967    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"]
Sep 10 23:58:13.265321 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:58:13.265295    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0d9fb1_8d59_4f6b_bc88_6097657cb5bc.slice/crio-40fddd2e8f9108b4ebeb12e5c993c0d85fafa22fecf1ba85c040d5557deb9506 WatchSource:0}: Error finding container 40fddd2e8f9108b4ebeb12e5c993c0d85fafa22fecf1ba85c040d5557deb9506: Status 404 returned error can't find the container with id 40fddd2e8f9108b4ebeb12e5c993c0d85fafa22fecf1ba85c040d5557deb9506
Sep 10 23:58:13.267592 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:13.267560    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:58:13.267677 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:13.267629    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:58:13.267677 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:13.267654    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:58:14.174563 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:14.174521    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" event={"ID":"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc","Type":"ContainerStarted","Data":"e077f9bf6b3bca95b0ee73b2cbabc5166756293b880bd27e301ec727702bbadc"}
Sep 10 23:58:14.174999 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:14.174579    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" event={"ID":"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc","Type":"ContainerStarted","Data":"40fddd2e8f9108b4ebeb12e5c993c0d85fafa22fecf1ba85c040d5557deb9506"}
Sep 10 23:58:14.192778 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:14.192726    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" podStartSLOduration=2.192707804 podStartE2EDuration="2.192707804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:58:14.191754999 +0000 UTC m=+1507.845475173" watchObservedRunningTime="2026-09-10 23:58:14.192707804 +0000 UTC m=+1507.846427978"
Sep 10 23:58:14.987301 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:14.987270    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:14.992273 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:14.992250    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:15.180111 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:15.180078    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:15.181198 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:15.181175    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:17.905038 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.905003    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"]
Sep 10 23:58:17.905498 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.905402    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy"
Sep 10 23:58:17.905498 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.905412    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ae0aea-b104-43e7-8435-f9087ec99fb4" containerName="istio-proxy"
Sep 10 23:58:17.908530 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.908515    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:17.912580 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.912554    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-71c1cb\""
Sep 10 23:58:17.913025 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.913007    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-71c1cb-dockercfg-rkxb5\""
Sep 10 23:58:17.923910 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.923888    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"]
Sep 10 23:58:17.934608 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.934583    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvg2b\" (UniqueName: \"kubernetes.io/projected/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-kube-api-access-vvg2b\") pod \"payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954\" (UID: \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:17.934737 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.934681    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954\" (UID: \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:17.941150 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.941128    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"]
Sep 10 23:58:17.944503 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.944487    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:17.954556 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:17.954533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"]
Sep 10 23:58:18.035238 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.035204    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954\" (UID: \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:18.035422 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.035258    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d4zh\" (UniqueName: \"kubernetes.io/projected/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-kube-api-access-6d4zh\") pod \"payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj\" (UID: \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:18.035422 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.035283    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj\" (UID: \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:18.035422 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.035338    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvg2b\" (UniqueName: \"kubernetes.io/projected/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-kube-api-access-vvg2b\") pod \"payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954\" (UID: \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:18.035916 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.035897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954\" (UID: \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:18.042471 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.042448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvg2b\" (UniqueName: \"kubernetes.io/projected/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-kube-api-access-vvg2b\") pod \"payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954\" (UID: \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:18.136595 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.136550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d4zh\" (UniqueName: \"kubernetes.io/projected/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-kube-api-access-6d4zh\") pod \"payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj\" (UID: \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:18.136595 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.136602    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj\" (UID: \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:18.137297 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.137276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj\" (UID: \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:18.143864 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.143834    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d4zh\" (UniqueName: \"kubernetes.io/projected/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-kube-api-access-6d4zh\") pod \"payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj\" (UID: \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:18.219422 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.219328    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:18.255979 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.255922    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:18.491958 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.491910    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"]
Sep 10 23:58:18.493979 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:58:18.493929    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f18d65_3ed2_4890_a6fe_a5d7f37b7662.slice/crio-4c04ba121b5a2c26464bd4648becf73f49814d31298b1d2c69d36cc23ebba174 WatchSource:0}: Error finding container 4c04ba121b5a2c26464bd4648becf73f49814d31298b1d2c69d36cc23ebba174: Status 404 returned error can't find the container with id 4c04ba121b5a2c26464bd4648becf73f49814d31298b1d2c69d36cc23ebba174
Sep 10 23:58:18.541480 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:18.541458    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"]
Sep 10 23:58:18.543365 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:58:18.543340    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb3cbd3_f8b3_4cdf_b644_d5e5b4665c17.slice/crio-45d7ff88952e6a6b5ad16ee27eba36c2f3b201a4608f5e6ad14f7ccb1aacaba8 WatchSource:0}: Error finding container 45d7ff88952e6a6b5ad16ee27eba36c2f3b201a4608f5e6ad14f7ccb1aacaba8: Status 404 returned error can't find the container with id 45d7ff88952e6a6b5ad16ee27eba36c2f3b201a4608f5e6ad14f7ccb1aacaba8
Sep 10 23:58:19.203449 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:19.203412    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj" event={"ID":"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17","Type":"ContainerStarted","Data":"5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0"}
Sep 10 23:58:19.203892 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:19.203465    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj" event={"ID":"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17","Type":"ContainerStarted","Data":"45d7ff88952e6a6b5ad16ee27eba36c2f3b201a4608f5e6ad14f7ccb1aacaba8"}
Sep 10 23:58:19.203892 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:19.203522    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:19.205207 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:19.205180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954" event={"ID":"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662","Type":"ContainerStarted","Data":"99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af"}
Sep 10 23:58:19.205356 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:19.205235    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:19.205356 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:19.205258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954" event={"ID":"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662","Type":"ContainerStarted","Data":"4c04ba121b5a2c26464bd4648becf73f49814d31298b1d2c69d36cc23ebba174"}
Sep 10 23:58:19.220430 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:19.220380    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj" podStartSLOduration=2.220364015 podStartE2EDuration="2.220364015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:58:19.21962816 +0000 UTC m=+1512.873348333" watchObservedRunningTime="2026-09-10 23:58:19.220364015 +0000 UTC m=+1512.874084191"
Sep 10 23:58:19.237030 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:19.236965    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954" podStartSLOduration=2.236928315 podStartE2EDuration="2.236928315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:58:19.233495307 +0000 UTC m=+1512.887215483" watchObservedRunningTime="2026-09-10 23:58:19.236928315 +0000 UTC m=+1512.890648489"
Sep 10 23:58:30.212628 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:30.212595    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 10 23:58:30.213035 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:30.212762    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 10 23:58:31.222120 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:31.222077    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 23:58:31.222496 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:31.222140    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 23:58:33.308669 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:33.308637    2569 generic.go:356] "Generic (PLEG): container finished" podID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerID="e077f9bf6b3bca95b0ee73b2cbabc5166756293b880bd27e301ec727702bbadc" exitCode=0
Sep 10 23:58:33.308669 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:33.308674    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" event={"ID":"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc","Type":"ContainerDied","Data":"e077f9bf6b3bca95b0ee73b2cbabc5166756293b880bd27e301ec727702bbadc"}
Sep 10 23:58:33.309273 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:33.309093    2569 scope.go:117] "RemoveContainer" containerID="e077f9bf6b3bca95b0ee73b2cbabc5166756293b880bd27e301ec727702bbadc"
Sep 10 23:58:33.310472 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:33.310439    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:58:33.310614 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:33.310592    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:58:33.310665 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:33.310648    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 10 23:58:34.316434 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:34.316399    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" event={"ID":"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc","Type":"ContainerStarted","Data":"2b81ca9f0eea5feb7a3873bf804c68b6b3a2e6a0aebc1a565325836670791144"}
Sep 10 23:58:34.987125 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:34.987093    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:34.987373 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:34.987353    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 10 23:58:34.987420 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:34.987399    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 10 23:58:35.987565 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:35.987530    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 10 23:58:35.987978 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:35.987582    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 10 23:58:36.987037 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:36.987001    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 10 23:58:36.987301 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:36.987058    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 10 23:58:37.987358 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:37.987318    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 10 23:58:37.987744 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:37.987380    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 10 23:58:38.987977 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:38.987923    2569 patch_prober.go:28] interesting pod/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 10 23:58:38.988358 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:38.988001    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 10 23:58:40.003782 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:40.003739    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:40.004243 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:40.004155    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:40.004917 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:40.004900    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 10 23:58:47.105056 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.105020    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"]
Sep 10 23:58:47.108733 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.108717    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:47.111414 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.111393    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-5453e3\""
Sep 10 23:58:47.111532 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.111433    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-5453e3-dockercfg-m8q5j\""
Sep 10 23:58:47.120170 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.120145    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"]
Sep 10 23:58:47.142811 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.142780    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"]
Sep 10 23:58:47.146205 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.146187    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:47.156830 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.156805    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"]
Sep 10 23:58:47.227049 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.227006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9nxv\" (UniqueName: \"kubernetes.io/projected/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-kube-api-access-x9nxv\") pod \"payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv\" (UID: \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:47.227244 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.227130    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv\" (UID: \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:47.328357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.328307    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mwq9\" (UniqueName: \"kubernetes.io/projected/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-kube-api-access-4mwq9\") pod \"payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k\" (UID: \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:47.328357 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.328364    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k\" (UID: \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:47.328583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.328438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9nxv\" (UniqueName: \"kubernetes.io/projected/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-kube-api-access-x9nxv\") pod \"payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv\" (UID: \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:47.328583 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.328495    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv\" (UID: \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:47.329199 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.329180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv\" (UID: \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:47.336999 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.336968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9nxv\" (UniqueName: \"kubernetes.io/projected/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-kube-api-access-x9nxv\") pod \"payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv\" (UID: \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:47.420654 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.420622    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:47.429413 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.429380    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k\" (UID: \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:47.429578 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.429549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mwq9\" (UniqueName: \"kubernetes.io/projected/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-kube-api-access-4mwq9\") pod \"payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k\" (UID: \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:47.430097 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.430073    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k\" (UID: \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:47.437163 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.437135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mwq9\" (UniqueName: \"kubernetes.io/projected/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-kube-api-access-4mwq9\") pod \"payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k\" (UID: \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:47.457071 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.457036    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:47.708082 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.708049    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"]
Sep 10 23:58:47.709783 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:58:47.709755    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5fceff9_c83e_40a7_ac8b_35cbf1ee393b.slice/crio-fdf3a95a7e88ab21eaa0a1fee69d8b1186280021e57112af2b41c1d8c8570b69 WatchSource:0}: Error finding container fdf3a95a7e88ab21eaa0a1fee69d8b1186280021e57112af2b41c1d8c8570b69: Status 404 returned error can't find the container with id fdf3a95a7e88ab21eaa0a1fee69d8b1186280021e57112af2b41c1d8c8570b69
Sep 10 23:58:47.736092 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:47.736071    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"]
Sep 10 23:58:47.740165 ip-10-0-143-154 kubenswrapper[2569]: W0910 23:58:47.740137    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e148dc_ecb2_49f2_bbed_0bf2aef5621b.slice/crio-627b8ba53bd0d2c72ccf07722814aa644a3bde904f597d83b03affd23559edd6 WatchSource:0}: Error finding container 627b8ba53bd0d2c72ccf07722814aa644a3bde904f597d83b03affd23559edd6: Status 404 returned error can't find the container with id 627b8ba53bd0d2c72ccf07722814aa644a3bde904f597d83b03affd23559edd6
Sep 10 23:58:48.398559 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:48.398508    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv" event={"ID":"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b","Type":"ContainerStarted","Data":"5801aa0b45070057fd2ea05e36c1d5e578e04bc721d5ecca3a9745805dae6550"}
Sep 10 23:58:48.399235 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:48.398578    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv" event={"ID":"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b","Type":"ContainerStarted","Data":"fdf3a95a7e88ab21eaa0a1fee69d8b1186280021e57112af2b41c1d8c8570b69"}
Sep 10 23:58:48.399235 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:48.398707    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:48.400127 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:48.400106    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k" event={"ID":"82e148dc-ecb2-49f2-bbed-0bf2aef5621b","Type":"ContainerStarted","Data":"1548e208decb4ff5f1334b401547ad222f6cc6e35ea1d5662c86ab78558573db"}
Sep 10 23:58:48.400233 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:48.400147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k" event={"ID":"82e148dc-ecb2-49f2-bbed-0bf2aef5621b","Type":"ContainerStarted","Data":"627b8ba53bd0d2c72ccf07722814aa644a3bde904f597d83b03affd23559edd6"}
Sep 10 23:58:48.400290 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:48.400242    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:58:48.415261 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:48.415223    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv" podStartSLOduration=1.415211849 podStartE2EDuration="1.415211849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:58:48.413217836 +0000 UTC m=+1542.066938009" watchObservedRunningTime="2026-09-10 23:58:48.415211849 +0000 UTC m=+1542.068932022"
Sep 10 23:58:48.428133 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:48.428096    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k" podStartSLOduration=1.428083249 podStartE2EDuration="1.428083249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 23:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 23:58:48.427150962 +0000 UTC m=+1542.080871159" watchObservedRunningTime="2026-09-10 23:58:48.428083249 +0000 UTC m=+1542.081803449"
Sep 10 23:58:59.407828 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:59.407796    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 10 23:58:59.408291 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:58:59.407870    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 10 23:59:15.433296 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:59:15.433265    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k_82e148dc-ecb2-49f2-bbed-0bf2aef5621b/payload-processing/0.log"
Sep 10 23:59:56.039268 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:59:56.039231    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj_ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17/payload-processing/0.log"
Sep 10 23:59:56.264979 ip-10-0-143-154 kubenswrapper[2569]: I0910 23:59:56.264928    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k_82e148dc-ecb2-49f2-bbed-0bf2aef5621b/payload-processing/0.log"
Sep 11 00:00:00.206165 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.206082    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm"]
Sep 11 00:00:00.210054 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.210019    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"]
Sep 11 00:00:00.210209 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.210168    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm"
Sep 11 00:00:00.212901 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.212879    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 11 00:00:00.213396 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.213382    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"
Sep 11 00:00:00.235924 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.235896    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm"]
Sep 11 00:00:00.242752 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.242686    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"]
Sep 11 00:00:00.245318 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.245293    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm"
Sep 11 00:00:00.253387 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.253367    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"
Sep 11 00:00:00.522400 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.522375    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm"]
Sep 11 00:00:00.524386 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:00:00.524358    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80484705_6646_4448_8fff_4bbb9649e77f.slice/crio-4ca59d36fca2a03965211f4fb8f54c9c80ba05a8b0ed7c84285b86a97771e353 WatchSource:0}: Error finding container 4ca59d36fca2a03965211f4fb8f54c9c80ba05a8b0ed7c84285b86a97771e353: Status 404 returned error can't find the container with id 4ca59d36fca2a03965211f4fb8f54c9c80ba05a8b0ed7c84285b86a97771e353
Sep 11 00:00:00.540129 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.540101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"]
Sep 11 00:00:00.542487 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:00:00.542447    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63019e8c_dfdd_48d7_8383_e2d3593e46ba.slice/crio-3cdb5a2d8fb09d011ed531237482f8148ef8dd75b8249f2997dbe333e170d8bf WatchSource:0}: Error finding container 3cdb5a2d8fb09d011ed531237482f8148ef8dd75b8249f2997dbe333e170d8bf: Status 404 returned error can't find the container with id 3cdb5a2d8fb09d011ed531237482f8148ef8dd75b8249f2997dbe333e170d8bf
Sep 11 00:00:00.792388 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.792292    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2" event={"ID":"63019e8c-dfdd-48d7-8383-e2d3593e46ba","Type":"ContainerStarted","Data":"3cdb5a2d8fb09d011ed531237482f8148ef8dd75b8249f2997dbe333e170d8bf"}
Sep 11 00:00:00.793536 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:00.793512    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm" event={"ID":"80484705-6646-4448-8fff-4bbb9649e77f","Type":"ContainerStarted","Data":"4ca59d36fca2a03965211f4fb8f54c9c80ba05a8b0ed7c84285b86a97771e353"}
Sep 11 00:00:01.642748 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.642715    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"]
Sep 11 00:00:01.646464 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.646437    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:01.649071 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.649045    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-8405fd\""
Sep 11 00:00:01.649198 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.649160    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-8405fd-dockercfg-qlgcg\""
Sep 11 00:00:01.656804 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.656780    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"]
Sep 11 00:00:01.742388 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.742289    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85937a8-320a-473a-8143-22015b50e3ec-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr\" (UID: \"d85937a8-320a-473a-8143-22015b50e3ec\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:01.742548 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.742465    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-456kt\" (UniqueName: \"kubernetes.io/projected/d85937a8-320a-473a-8143-22015b50e3ec-kube-api-access-456kt\") pod \"payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr\" (UID: \"d85937a8-320a-473a-8143-22015b50e3ec\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:01.801849 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.801810    2569 generic.go:356] "Generic (PLEG): container finished" podID="63019e8c-dfdd-48d7-8383-e2d3593e46ba" containerID="66f428a6ed5e55121ac15d5728ecf266853ed46ed1dd51b3a3d833c759faf921" exitCode=0
Sep 11 00:00:01.802057 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.801890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2" event={"ID":"63019e8c-dfdd-48d7-8383-e2d3593e46ba","Type":"ContainerDied","Data":"66f428a6ed5e55121ac15d5728ecf266853ed46ed1dd51b3a3d833c759faf921"}
Sep 11 00:00:01.803552 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.803513    2569 generic.go:356] "Generic (PLEG): container finished" podID="80484705-6646-4448-8fff-4bbb9649e77f" containerID="b7206e42984c0c030395b7d6d4416ae5575fc30681f0a213d4c1bfe3c1b41f92" exitCode=0
Sep 11 00:00:01.803658 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.803593    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm" event={"ID":"80484705-6646-4448-8fff-4bbb9649e77f","Type":"ContainerDied","Data":"b7206e42984c0c030395b7d6d4416ae5575fc30681f0a213d4c1bfe3c1b41f92"}
Sep 11 00:00:01.843979 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.843920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-456kt\" (UniqueName: \"kubernetes.io/projected/d85937a8-320a-473a-8143-22015b50e3ec-kube-api-access-456kt\") pod \"payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr\" (UID: \"d85937a8-320a-473a-8143-22015b50e3ec\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:01.844173 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.844042    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85937a8-320a-473a-8143-22015b50e3ec-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr\" (UID: \"d85937a8-320a-473a-8143-22015b50e3ec\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:01.844787 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.844768    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85937a8-320a-473a-8143-22015b50e3ec-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr\" (UID: \"d85937a8-320a-473a-8143-22015b50e3ec\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:01.851708 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.851681    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-456kt\" (UniqueName: \"kubernetes.io/projected/d85937a8-320a-473a-8143-22015b50e3ec-kube-api-access-456kt\") pod \"payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr\" (UID: \"d85937a8-320a-473a-8143-22015b50e3ec\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:01.960118 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:01.960075    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:02.230604 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:02.230577    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"]
Sep 11 00:00:02.232190 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:00:02.232167    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85937a8_320a_473a_8143_22015b50e3ec.slice/crio-edab8057b0e5a569bcb0167dd0cd95438013c6a02d14892f6c818e0f96695bf1 WatchSource:0}: Error finding container edab8057b0e5a569bcb0167dd0cd95438013c6a02d14892f6c818e0f96695bf1: Status 404 returned error can't find the container with id edab8057b0e5a569bcb0167dd0cd95438013c6a02d14892f6c818e0f96695bf1
Sep 11 00:00:02.811301 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:02.811260    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr" event={"ID":"d85937a8-320a-473a-8143-22015b50e3ec","Type":"ContainerStarted","Data":"59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836"}
Sep 11 00:00:02.811778 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:02.811331    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:02.811778 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:02.811355    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr" event={"ID":"d85937a8-320a-473a-8143-22015b50e3ec","Type":"ContainerStarted","Data":"edab8057b0e5a569bcb0167dd0cd95438013c6a02d14892f6c818e0f96695bf1"}
Sep 11 00:00:02.831223 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:02.831155    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr" podStartSLOduration=1.831131014 podStartE2EDuration="1.831131014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:00:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:00:02.826253341 +0000 UTC m=+1616.479973514" watchObservedRunningTime="2026-09-11 00:00:02.831131014 +0000 UTC m=+1616.484851267"
Sep 11 00:00:02.974416 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:02.974389    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"
Sep 11 00:00:03.011779 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:03.011756    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm"
Sep 11 00:00:03.818315 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:03.818285    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"
Sep 11 00:00:03.818315 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:03.818308    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2" event={"ID":"63019e8c-dfdd-48d7-8383-e2d3593e46ba","Type":"ContainerDied","Data":"3cdb5a2d8fb09d011ed531237482f8148ef8dd75b8249f2997dbe333e170d8bf"}
Sep 11 00:00:03.818781 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:03.818358    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3cdb5a2d8fb09d011ed531237482f8148ef8dd75b8249f2997dbe333e170d8bf"
Sep 11 00:00:03.819849 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:03.819831    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm"
Sep 11 00:00:03.819962 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:03.819883    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818080-rlzxm" event={"ID":"80484705-6646-4448-8fff-4bbb9649e77f","Type":"ContainerDied","Data":"4ca59d36fca2a03965211f4fb8f54c9c80ba05a8b0ed7c84285b86a97771e353"}
Sep 11 00:00:03.820002 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:03.819957    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ca59d36fca2a03965211f4fb8f54c9c80ba05a8b0ed7c84285b86a97771e353"
Sep 11 00:00:07.332036 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:07.332003    2569 scope.go:117] "RemoveContainer" containerID="76bcc70f737e8e7a6559bb576e031d106c50b96e4bfd20be5c78fb446ba7c81f"
Sep 11 00:00:13.822125 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:13.822082    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:31.549478 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.549438    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"]
Sep 11 00:00:31.550040 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.549673    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr" podUID="d85937a8-320a-473a-8143-22015b50e3ec" containerName="payload-processing" containerID="cri-o://59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836" gracePeriod=30
Sep 11 00:00:31.815475 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.815418    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:31.948158 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.948122    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-456kt\" (UniqueName: \"kubernetes.io/projected/d85937a8-320a-473a-8143-22015b50e3ec-kube-api-access-456kt\") pod \"d85937a8-320a-473a-8143-22015b50e3ec\" (UID: \"d85937a8-320a-473a-8143-22015b50e3ec\") "
Sep 11 00:00:31.948361 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.948236    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85937a8-320a-473a-8143-22015b50e3ec-plugins-config-volume\") pod \"d85937a8-320a-473a-8143-22015b50e3ec\" (UID: \"d85937a8-320a-473a-8143-22015b50e3ec\") "
Sep 11 00:00:31.948610 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.948585    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d85937a8-320a-473a-8143-22015b50e3ec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d85937a8-320a-473a-8143-22015b50e3ec" (UID: "d85937a8-320a-473a-8143-22015b50e3ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:00:31.950287 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.950260    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d85937a8-320a-473a-8143-22015b50e3ec-kube-api-access-456kt" (OuterVolumeSpecName: "kube-api-access-456kt") pod "d85937a8-320a-473a-8143-22015b50e3ec" (UID: "d85937a8-320a-473a-8143-22015b50e3ec"). InnerVolumeSpecName "kube-api-access-456kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:00:31.973861 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.973830    2569 generic.go:356] "Generic (PLEG): container finished" podID="d85937a8-320a-473a-8143-22015b50e3ec" containerID="59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836" exitCode=0
Sep 11 00:00:31.973861 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.973863    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"
Sep 11 00:00:31.974058 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.973893    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr" event={"ID":"d85937a8-320a-473a-8143-22015b50e3ec","Type":"ContainerDied","Data":"59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836"}
Sep 11 00:00:31.974058 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.973929    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr" event={"ID":"d85937a8-320a-473a-8143-22015b50e3ec","Type":"ContainerDied","Data":"edab8057b0e5a569bcb0167dd0cd95438013c6a02d14892f6c818e0f96695bf1"}
Sep 11 00:00:31.974058 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.973976    2569 scope.go:117] "RemoveContainer" containerID="59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836"
Sep 11 00:00:31.983628 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.983607    2569 scope.go:117] "RemoveContainer" containerID="59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836"
Sep 11 00:00:31.983875 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:00:31.983853    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836\": container with ID starting with 59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836 not found: ID does not exist" containerID="59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836"
Sep 11 00:00:31.983948 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.983880    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836"} err="failed to get container status \"59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836\": rpc error: code = NotFound desc = could not find container \"59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836\": container with ID starting with 59ae2f1df38ffc24ba8682d474417add68a618e0e99fc950b746bdef4efdc836 not found: ID does not exist"
Sep 11 00:00:31.994061 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.994038    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"]
Sep 11 00:00:31.998367 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:31.998346    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-8405fd-59d8f97f7c-j6ftr"]
Sep 11 00:00:32.049560 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:32.049527    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d85937a8-320a-473a-8143-22015b50e3ec-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:00:32.049560 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:32.049553    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-456kt\" (UniqueName: \"kubernetes.io/projected/d85937a8-320a-473a-8143-22015b50e3ec-kube-api-access-456kt\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:00:32.979289 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:32.979254    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d85937a8-320a-473a-8143-22015b50e3ec" path="/var/lib/kubelet/pods/d85937a8-320a-473a-8143-22015b50e3ec/volumes"
Sep 11 00:00:41.605457 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605420    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"]
Sep 11 00:00:41.605904 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605822    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63019e8c-dfdd-48d7-8383-e2d3593e46ba" containerName="cleanup"
Sep 11 00:00:41.605904 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605833    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="63019e8c-dfdd-48d7-8383-e2d3593e46ba" containerName="cleanup"
Sep 11 00:00:41.605904 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605842    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80484705-6646-4448-8fff-4bbb9649e77f" containerName="cleanup"
Sep 11 00:00:41.605904 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605848    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="80484705-6646-4448-8fff-4bbb9649e77f" containerName="cleanup"
Sep 11 00:00:41.605904 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605875    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d85937a8-320a-473a-8143-22015b50e3ec" containerName="payload-processing"
Sep 11 00:00:41.605904 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605882    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d85937a8-320a-473a-8143-22015b50e3ec" containerName="payload-processing"
Sep 11 00:00:41.606172 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605962    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="63019e8c-dfdd-48d7-8383-e2d3593e46ba" containerName="cleanup"
Sep 11 00:00:41.606172 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605972    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d85937a8-320a-473a-8143-22015b50e3ec" containerName="payload-processing"
Sep 11 00:00:41.606172 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.605980    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="80484705-6646-4448-8fff-4bbb9649e77f" containerName="cleanup"
Sep 11 00:00:41.608860 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.608844    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"
Sep 11 00:00:41.610748 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.610726    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 00:00:41.610860 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.610828    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 11 00:00:41.614815 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.614788    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"]
Sep 11 00:00:41.744340 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.744302    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c5003c-8805-4ee3-82da-594e8b6b290a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn\" (UID: \"e8c5003c-8805-4ee3-82da-594e8b6b290a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"
Sep 11 00:00:41.845891 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.845856    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c5003c-8805-4ee3-82da-594e8b6b290a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn\" (UID: \"e8c5003c-8805-4ee3-82da-594e8b6b290a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"
Sep 11 00:00:41.846532 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.846511    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c5003c-8805-4ee3-82da-594e8b6b290a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn\" (UID: \"e8c5003c-8805-4ee3-82da-594e8b6b290a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"
Sep 11 00:00:41.921624 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:41.921588    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"
Sep 11 00:00:42.192434 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:42.192407    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"]
Sep 11 00:00:42.194590 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:00:42.194563    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c5003c_8805_4ee3_82da_594e8b6b290a.slice/crio-7504c7f57dbe33db8cc331a0b845ad9d175af03d9c044fc7ba5e099c236b36c7 WatchSource:0}: Error finding container 7504c7f57dbe33db8cc331a0b845ad9d175af03d9c044fc7ba5e099c236b36c7: Status 404 returned error can't find the container with id 7504c7f57dbe33db8cc331a0b845ad9d175af03d9c044fc7ba5e099c236b36c7
Sep 11 00:00:43.035319 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:43.035289    2569 generic.go:356] "Generic (PLEG): container finished" podID="e8c5003c-8805-4ee3-82da-594e8b6b290a" containerID="3b2a79dd2de0b01dbec1ab4d4f5ae00e59bb56cd6c890ff9c0905047dd21e60e" exitCode=0
Sep 11 00:00:43.035712 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:43.035332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn" event={"ID":"e8c5003c-8805-4ee3-82da-594e8b6b290a","Type":"ContainerDied","Data":"3b2a79dd2de0b01dbec1ab4d4f5ae00e59bb56cd6c890ff9c0905047dd21e60e"}
Sep 11 00:00:43.035712 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:43.035372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn" event={"ID":"e8c5003c-8805-4ee3-82da-594e8b6b290a","Type":"ContainerStarted","Data":"7504c7f57dbe33db8cc331a0b845ad9d175af03d9c044fc7ba5e099c236b36c7"}
Sep 11 00:00:44.188889 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:44.188863    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"
Sep 11 00:00:44.369590 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:44.369558    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c5003c-8805-4ee3-82da-594e8b6b290a-maas-api-service-ca\") pod \"e8c5003c-8805-4ee3-82da-594e8b6b290a\" (UID: \"e8c5003c-8805-4ee3-82da-594e8b6b290a\") "
Sep 11 00:00:44.369928 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:44.369905    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c5003c-8805-4ee3-82da-594e8b6b290a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8c5003c-8805-4ee3-82da-594e8b6b290a" (UID: "e8c5003c-8805-4ee3-82da-594e8b6b290a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:00:44.470801 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:44.470762    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8c5003c-8805-4ee3-82da-594e8b6b290a-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:00:45.047185 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:45.047153    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn" event={"ID":"e8c5003c-8805-4ee3-82da-594e8b6b290a","Type":"ContainerDied","Data":"7504c7f57dbe33db8cc331a0b845ad9d175af03d9c044fc7ba5e099c236b36c7"}
Sep 11 00:00:45.047185 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:45.047189    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"
Sep 11 00:00:45.047393 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:45.047202    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7504c7f57dbe33db8cc331a0b845ad9d175af03d9c044fc7ba5e099c236b36c7"
Sep 11 00:00:57.060294 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.060254    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"]
Sep 11 00:00:57.060890 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.060509    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj" podUID="ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17" containerName="payload-processing" containerID="cri-o://5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0" gracePeriod=30
Sep 11 00:00:57.068912 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.068520    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"]
Sep 11 00:00:57.068912 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.068766    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954" podUID="b7f18d65-3ed2-4890-a6fe-a5d7f37b7662" containerName="payload-pre-processing" containerID="cri-o://99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af" gracePeriod=30
Sep 11 00:00:57.210512 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.210477    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"]
Sep 11 00:00:57.213281 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.213251    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-71c1cb-576qn"]
Sep 11 00:00:57.356114 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.356078    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 11 00:00:57.358875 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.358857    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 11 00:00:57.397057 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.395539    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-plugins-config-volume\") pod \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\" (UID: \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\") "
Sep 11 00:00:57.397057 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.395598    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d4zh\" (UniqueName: \"kubernetes.io/projected/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-kube-api-access-6d4zh\") pod \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\" (UID: \"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17\") "
Sep 11 00:00:57.397057 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.395649    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-plugins-config-volume\") pod \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\" (UID: \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\") "
Sep 11 00:00:57.397057 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.395726    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvg2b\" (UniqueName: \"kubernetes.io/projected/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-kube-api-access-vvg2b\") pod \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\" (UID: \"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662\") "
Sep 11 00:00:57.397057 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.396688    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17" (UID: "ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:00:57.397057 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.396738    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7f18d65-3ed2-4890-a6fe-a5d7f37b7662" (UID: "b7f18d65-3ed2-4890-a6fe-a5d7f37b7662"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:00:57.400091 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.400047    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-kube-api-access-vvg2b" (OuterVolumeSpecName: "kube-api-access-vvg2b") pod "b7f18d65-3ed2-4890-a6fe-a5d7f37b7662" (UID: "b7f18d65-3ed2-4890-a6fe-a5d7f37b7662"). InnerVolumeSpecName "kube-api-access-vvg2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:00:57.401090 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.401066    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-kube-api-access-6d4zh" (OuterVolumeSpecName: "kube-api-access-6d4zh") pod "ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17" (UID: "ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17"). InnerVolumeSpecName "kube-api-access-6d4zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:00:57.496784 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.496748    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:00:57.496784 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.496778    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d4zh\" (UniqueName: \"kubernetes.io/projected/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17-kube-api-access-6d4zh\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:00:57.496784 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.496789    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:00:57.497068 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:57.496797    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvg2b\" (UniqueName: \"kubernetes.io/projected/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662-kube-api-access-vvg2b\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:00:58.117795 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.117761    2569 generic.go:356] "Generic (PLEG): container finished" podID="b7f18d65-3ed2-4890-a6fe-a5d7f37b7662" containerID="99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af" exitCode=0
Sep 11 00:00:58.118299 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.117814    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"
Sep 11 00:00:58.118299 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.117857    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954" event={"ID":"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662","Type":"ContainerDied","Data":"99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af"}
Sep 11 00:00:58.118299 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.117912    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954" event={"ID":"b7f18d65-3ed2-4890-a6fe-a5d7f37b7662","Type":"ContainerDied","Data":"4c04ba121b5a2c26464bd4648becf73f49814d31298b1d2c69d36cc23ebba174"}
Sep 11 00:00:58.118299 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.117956    2569 scope.go:117] "RemoveContainer" containerID="99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af"
Sep 11 00:00:58.119273 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.119244    2569 generic.go:356] "Generic (PLEG): container finished" podID="ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17" containerID="5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0" exitCode=0
Sep 11 00:00:58.119347 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.119281    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"
Sep 11 00:00:58.119406 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.119281    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj" event={"ID":"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17","Type":"ContainerDied","Data":"5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0"}
Sep 11 00:00:58.119406 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.119395    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj" event={"ID":"ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17","Type":"ContainerDied","Data":"45d7ff88952e6a6b5ad16ee27eba36c2f3b201a4608f5e6ad14f7ccb1aacaba8"}
Sep 11 00:00:58.133281 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.133160    2569 scope.go:117] "RemoveContainer" containerID="99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af"
Sep 11 00:00:58.134271 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:00:58.133692    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af\": container with ID starting with 99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af not found: ID does not exist" containerID="99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af"
Sep 11 00:00:58.134271 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.133750    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af"} err="failed to get container status \"99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af\": rpc error: code = NotFound desc = could not find container \"99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af\": container with ID starting with 99e10306e465d9f6349d9b74bb9424155de361b928debc2d64ec4c653cb306af not found: ID does not exist"
Sep 11 00:00:58.134271 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.133778    2569 scope.go:117] "RemoveContainer" containerID="5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0"
Sep 11 00:00:58.145239 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.145221    2569 scope.go:117] "RemoveContainer" containerID="5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0"
Sep 11 00:00:58.145505 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:00:58.145486    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0\": container with ID starting with 5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0 not found: ID does not exist" containerID="5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0"
Sep 11 00:00:58.145554 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.145512    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0"} err="failed to get container status \"5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0\": rpc error: code = NotFound desc = could not find container \"5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0\": container with ID starting with 5e8f227bd9977782907902237c7bd69b556261e7edeb131c5ad8169a0adf2ad0 not found: ID does not exist"
Sep 11 00:00:58.149190 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.149168    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"]
Sep 11 00:00:58.152702 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.152679    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-71c1cb-c58f5b878-f4954"]
Sep 11 00:00:58.161368 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.161347    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"]
Sep 11 00:00:58.165348 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.165329    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-71c1cb-559c98dc4c-hggqj"]
Sep 11 00:00:58.979048 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.979014    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7f18d65-3ed2-4890-a6fe-a5d7f37b7662" path="/var/lib/kubelet/pods/b7f18d65-3ed2-4890-a6fe-a5d7f37b7662/volumes"
Sep 11 00:00:58.979392 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.979380    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17" path="/var/lib/kubelet/pods/ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17/volumes"
Sep 11 00:00:58.979671 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:00:58.979660    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8c5003c-8805-4ee3-82da-594e8b6b290a" path="/var/lib/kubelet/pods/e8c5003c-8805-4ee3-82da-594e8b6b290a/volumes"
Sep 11 00:01:59.329376 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.329334    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"]
Sep 11 00:01:59.329812 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.329578    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy" containerID="cri-o://2b81ca9f0eea5feb7a3873bf804c68b6b3a2e6a0aebc1a565325836670791144" gracePeriod=30
Sep 11 00:01:59.675602 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.675567    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"]
Sep 11 00:01:59.675991 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.675978    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17" containerName="payload-processing"
Sep 11 00:01:59.676045 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.675992    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17" containerName="payload-processing"
Sep 11 00:01:59.676045 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.676001    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7f18d65-3ed2-4890-a6fe-a5d7f37b7662" containerName="payload-pre-processing"
Sep 11 00:01:59.676045 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.676006    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7f18d65-3ed2-4890-a6fe-a5d7f37b7662" containerName="payload-pre-processing"
Sep 11 00:01:59.676045 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.676016    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c5003c-8805-4ee3-82da-594e8b6b290a" containerName="revoke-keys"
Sep 11 00:01:59.676045 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.676022    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c5003c-8805-4ee3-82da-594e8b6b290a" containerName="revoke-keys"
Sep 11 00:01:59.676202 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.676083    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7f18d65-3ed2-4890-a6fe-a5d7f37b7662" containerName="payload-pre-processing"
Sep 11 00:01:59.676202 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.676092    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddb3cbd3-f8b3-4cdf-b644-d5e5b4665c17" containerName="payload-processing"
Sep 11 00:01:59.676202 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.676098    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c5003c-8805-4ee3-82da-594e8b6b290a" containerName="revoke-keys"
Sep 11 00:01:59.679162 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.679146    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"
Sep 11 00:01:59.681469 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.681447    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 00:01:59.681582 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.681471    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 11 00:01:59.688602 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.688581    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"]
Sep 11 00:01:59.808540 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.808496    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e10e06-f09a-4693-a56e-ce7ecb3162a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn\" (UID: \"c8e10e06-f09a-4693-a56e-ce7ecb3162a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"
Sep 11 00:01:59.909411 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.909373    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e10e06-f09a-4693-a56e-ce7ecb3162a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn\" (UID: \"c8e10e06-f09a-4693-a56e-ce7ecb3162a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"
Sep 11 00:01:59.910026 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.910007    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e10e06-f09a-4693-a56e-ce7ecb3162a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn\" (UID: \"c8e10e06-f09a-4693-a56e-ce7ecb3162a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"
Sep 11 00:01:59.991372 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:01:59.991281    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"
Sep 11 00:02:00.267639 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:00.267606    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"]
Sep 11 00:02:00.269575 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:02:00.269550    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e10e06_f09a_4693_a56e_ce7ecb3162a4.slice/crio-5b731ebd846896a9522692d5f1263a4fa56aaa336046d77ba6caec5b91723e6d WatchSource:0}: Error finding container 5b731ebd846896a9522692d5f1263a4fa56aaa336046d77ba6caec5b91723e6d: Status 404 returned error can't find the container with id 5b731ebd846896a9522692d5f1263a4fa56aaa336046d77ba6caec5b91723e6d
Sep 11 00:02:00.463383 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:00.463349    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn" event={"ID":"c8e10e06-f09a-4693-a56e-ce7ecb3162a4","Type":"ContainerStarted","Data":"5b731ebd846896a9522692d5f1263a4fa56aaa336046d77ba6caec5b91723e6d"}
Sep 11 00:02:01.475012 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:01.474963    2569 generic.go:356] "Generic (PLEG): container finished" podID="c8e10e06-f09a-4693-a56e-ce7ecb3162a4" containerID="140fef472e7c526fe9996b6918c64fcd3e44293d184ced6cfbdcc6bf6bd67303" exitCode=0
Sep 11 00:02:01.475430 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:01.475122    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn" event={"ID":"c8e10e06-f09a-4693-a56e-ce7ecb3162a4","Type":"ContainerDied","Data":"140fef472e7c526fe9996b6918c64fcd3e44293d184ced6cfbdcc6bf6bd67303"}
Sep 11 00:02:02.612113 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:02.612089    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"
Sep 11 00:02:02.638072 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:02.638043    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e10e06-f09a-4693-a56e-ce7ecb3162a4-maas-api-service-ca\") pod \"c8e10e06-f09a-4693-a56e-ce7ecb3162a4\" (UID: \"c8e10e06-f09a-4693-a56e-ce7ecb3162a4\") "
Sep 11 00:02:02.638372 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:02.638349    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8e10e06-f09a-4693-a56e-ce7ecb3162a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c8e10e06-f09a-4693-a56e-ce7ecb3162a4" (UID: "c8e10e06-f09a-4693-a56e-ce7ecb3162a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:02.738824 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:02.738737    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8e10e06-f09a-4693-a56e-ce7ecb3162a4-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:03.487533 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:03.487502    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"
Sep 11 00:02:03.487711 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:03.487538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn" event={"ID":"c8e10e06-f09a-4693-a56e-ce7ecb3162a4","Type":"ContainerDied","Data":"5b731ebd846896a9522692d5f1263a4fa56aaa336046d77ba6caec5b91723e6d"}
Sep 11 00:02:03.487711 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:03.487585    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b731ebd846896a9522692d5f1263a4fa56aaa336046d77ba6caec5b91723e6d"
Sep 11 00:02:04.498053 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.498023    2569 generic.go:356] "Generic (PLEG): container finished" podID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerID="2b81ca9f0eea5feb7a3873bf804c68b6b3a2e6a0aebc1a565325836670791144" exitCode=0
Sep 11 00:02:04.498428 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.498098    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" event={"ID":"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc","Type":"ContainerDied","Data":"2b81ca9f0eea5feb7a3873bf804c68b6b3a2e6a0aebc1a565325836670791144"}
Sep 11 00:02:04.498428 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.498174    2569 scope.go:117] "RemoveContainer" containerID="e077f9bf6b3bca95b0ee73b2cbabc5166756293b880bd27e301ec727702bbadc"
Sep 11 00:02:04.594644 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.594582    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 11 00:02:04.655021 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.654990    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-data\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655194 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655027    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-socket\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655194 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655048    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-envoy\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655194 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655124    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrz6x\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-kube-api-access-lrz6x\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655194 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655147    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istiod-ca-cert\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655194 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655170    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-podinfo\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655194 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655190    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-certs\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655451 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655223    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-token\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655451 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655259    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-credential-socket\") pod \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\" (UID: \"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc\") "
Sep 11 00:02:04.655451 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655328    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:04.655451 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655364    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:04.655638 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655546    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:04.655638 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655550    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:04.655746 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655676    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:04.655746 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655694    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:04.655925 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.655894    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:04.657332 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.657310    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:04.657440 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.657426    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:04.657749 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.657726    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 00:02:04.657749 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.657740    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-kube-api-access-lrz6x" (OuterVolumeSpecName: "kube-api-access-lrz6x") pod "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" (UID: "bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc"). InnerVolumeSpecName "kube-api-access-lrz6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:04.757077 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.757036    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrz6x\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-kube-api-access-lrz6x\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:04.757077 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.757067    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:04.757077 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.757083    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:04.757328 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.757095    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:04.757328 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.757108    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:04.757328 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.757118    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:04.757328 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:04.757129    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:05.505518 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:05.505488    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"
Sep 11 00:02:05.505987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:05.505487    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4" event={"ID":"bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc","Type":"ContainerDied","Data":"40fddd2e8f9108b4ebeb12e5c993c0d85fafa22fecf1ba85c040d5557deb9506"}
Sep 11 00:02:05.505987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:05.505622    2569 scope.go:117] "RemoveContainer" containerID="2b81ca9f0eea5feb7a3873bf804c68b6b3a2e6a0aebc1a565325836670791144"
Sep 11 00:02:05.525087 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:05.525059    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"]
Sep 11 00:02:05.530500 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:05.530475    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-71c1cb-openshift-default-54b4b58868-t5np4"]
Sep 11 00:02:06.979619 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:06.979583    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" path="/var/lib/kubelet/pods/bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc/volumes"
Sep 11 00:02:10.370262 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.370230    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"]
Sep 11 00:02:10.370647 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.370486    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k" podUID="82e148dc-ecb2-49f2-bbed-0bf2aef5621b" containerName="payload-processing" containerID="cri-o://1548e208decb4ff5f1334b401547ad222f6cc6e35ea1d5662c86ab78558573db" gracePeriod=30
Sep 11 00:02:10.381508 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.381481    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"]
Sep 11 00:02:10.381682 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.381663    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv" podUID="f5fceff9-c83e-40a7-ac8b-35cbf1ee393b" containerName="payload-pre-processing" containerID="cri-o://5801aa0b45070057fd2ea05e36c1d5e578e04bc721d5ecca3a9745805dae6550" gracePeriod=30
Sep 11 00:02:10.535512 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.535470    2569 generic.go:356] "Generic (PLEG): container finished" podID="f5fceff9-c83e-40a7-ac8b-35cbf1ee393b" containerID="5801aa0b45070057fd2ea05e36c1d5e578e04bc721d5ecca3a9745805dae6550" exitCode=0
Sep 11 00:02:10.535701 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.535629    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv" event={"ID":"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b","Type":"ContainerDied","Data":"5801aa0b45070057fd2ea05e36c1d5e578e04bc721d5ecca3a9745805dae6550"}
Sep 11 00:02:10.537369 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.537344    2569 generic.go:356] "Generic (PLEG): container finished" podID="82e148dc-ecb2-49f2-bbed-0bf2aef5621b" containerID="1548e208decb4ff5f1334b401547ad222f6cc6e35ea1d5662c86ab78558573db" exitCode=0
Sep 11 00:02:10.537517 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.537414    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k" event={"ID":"82e148dc-ecb2-49f2-bbed-0bf2aef5621b","Type":"ContainerDied","Data":"1548e208decb4ff5f1334b401547ad222f6cc6e35ea1d5662c86ab78558573db"}
Sep 11 00:02:10.661802 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.661776    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"]
Sep 11 00:02:10.663858 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.663837    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 11 00:02:10.664919 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.664896    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5453e3-9h6rn"]
Sep 11 00:02:10.667481 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.667462    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 11 00:02:10.710200 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.710171    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9nxv\" (UniqueName: \"kubernetes.io/projected/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-kube-api-access-x9nxv\") pod \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\" (UID: \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\") "
Sep 11 00:02:10.710384 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.710211    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mwq9\" (UniqueName: \"kubernetes.io/projected/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-kube-api-access-4mwq9\") pod \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\" (UID: \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\") "
Sep 11 00:02:10.710384 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.710271    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-plugins-config-volume\") pod \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\" (UID: \"82e148dc-ecb2-49f2-bbed-0bf2aef5621b\") "
Sep 11 00:02:10.710384 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.710339    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-plugins-config-volume\") pod \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\" (UID: \"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b\") "
Sep 11 00:02:10.710676 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.710646    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82e148dc-ecb2-49f2-bbed-0bf2aef5621b" (UID: "82e148dc-ecb2-49f2-bbed-0bf2aef5621b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:10.710810 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.710679    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f5fceff9-c83e-40a7-ac8b-35cbf1ee393b" (UID: "f5fceff9-c83e-40a7-ac8b-35cbf1ee393b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:10.712317 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.712297    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-kube-api-access-x9nxv" (OuterVolumeSpecName: "kube-api-access-x9nxv") pod "f5fceff9-c83e-40a7-ac8b-35cbf1ee393b" (UID: "f5fceff9-c83e-40a7-ac8b-35cbf1ee393b"). InnerVolumeSpecName "kube-api-access-x9nxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:10.712431 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.712337    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-kube-api-access-4mwq9" (OuterVolumeSpecName: "kube-api-access-4mwq9") pod "82e148dc-ecb2-49f2-bbed-0bf2aef5621b" (UID: "82e148dc-ecb2-49f2-bbed-0bf2aef5621b"). InnerVolumeSpecName "kube-api-access-4mwq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:10.811242 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.811204    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:10.811242 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.811237    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9nxv\" (UniqueName: \"kubernetes.io/projected/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b-kube-api-access-x9nxv\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:10.811242 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.811247    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mwq9\" (UniqueName: \"kubernetes.io/projected/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-kube-api-access-4mwq9\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:10.811479 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.811255    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82e148dc-ecb2-49f2-bbed-0bf2aef5621b-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:10.977845 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:10.977761    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8e10e06-f09a-4693-a56e-ce7ecb3162a4" path="/var/lib/kubelet/pods/c8e10e06-f09a-4693-a56e-ce7ecb3162a4/volumes"
Sep 11 00:02:11.544915 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.544883    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"
Sep 11 00:02:11.545408 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.544918    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k" event={"ID":"82e148dc-ecb2-49f2-bbed-0bf2aef5621b","Type":"ContainerDied","Data":"627b8ba53bd0d2c72ccf07722814aa644a3bde904f597d83b03affd23559edd6"}
Sep 11 00:02:11.545408 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.545001    2569 scope.go:117] "RemoveContainer" containerID="1548e208decb4ff5f1334b401547ad222f6cc6e35ea1d5662c86ab78558573db"
Sep 11 00:02:11.546552 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.546536    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"
Sep 11 00:02:11.546630 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.546552    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv" event={"ID":"f5fceff9-c83e-40a7-ac8b-35cbf1ee393b","Type":"ContainerDied","Data":"fdf3a95a7e88ab21eaa0a1fee69d8b1186280021e57112af2b41c1d8c8570b69"}
Sep 11 00:02:11.554782 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.554765    2569 scope.go:117] "RemoveContainer" containerID="5801aa0b45070057fd2ea05e36c1d5e578e04bc721d5ecca3a9745805dae6550"
Sep 11 00:02:11.580678 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.580646    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"]
Sep 11 00:02:11.602990 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.602963    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-5453e3-7bfb7b8597-tw68k"]
Sep 11 00:02:11.614960 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.614920    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"]
Sep 11 00:02:11.617093 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:11.617072    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-5453e3-556bdb8fcd-7nrkv"]
Sep 11 00:02:12.977855 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:12.977824    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82e148dc-ecb2-49f2-bbed-0bf2aef5621b" path="/var/lib/kubelet/pods/82e148dc-ecb2-49f2-bbed-0bf2aef5621b/volumes"
Sep 11 00:02:12.978245 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:12.978208    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5fceff9-c83e-40a7-ac8b-35cbf1ee393b" path="/var/lib/kubelet/pods/f5fceff9-c83e-40a7-ac8b-35cbf1ee393b/volumes"
Sep 11 00:02:17.281321 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281282    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"]
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281664    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82e148dc-ecb2-49f2-bbed-0bf2aef5621b" containerName="payload-processing"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281681    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="82e148dc-ecb2-49f2-bbed-0bf2aef5621b" containerName="payload-processing"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281696    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281702    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281709    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5fceff9-c83e-40a7-ac8b-35cbf1ee393b" containerName="payload-pre-processing"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281715    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5fceff9-c83e-40a7-ac8b-35cbf1ee393b" containerName="payload-pre-processing"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281728    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281734    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281739    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8e10e06-f09a-4693-a56e-ce7ecb3162a4" containerName="revoke-keys"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281745    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8e10e06-f09a-4693-a56e-ce7ecb3162a4" containerName="revoke-keys"
Sep 11 00:02:17.281800 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281805    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy"
Sep 11 00:02:17.282162 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281815    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="82e148dc-ecb2-49f2-bbed-0bf2aef5621b" containerName="payload-processing"
Sep 11 00:02:17.282162 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281823    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8e10e06-f09a-4693-a56e-ce7ecb3162a4" containerName="revoke-keys"
Sep 11 00:02:17.282162 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.281831    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5fceff9-c83e-40a7-ac8b-35cbf1ee393b" containerName="payload-pre-processing"
Sep 11 00:02:17.286161 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.286145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"
Sep 11 00:02:17.288362 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.288340    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 00:02:17.288446 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.288340    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 11 00:02:17.291867 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.291843    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"]
Sep 11 00:02:17.375138 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.375100    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7368f6a9-46be-40c5-965b-78f2e3cfa301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j\" (UID: \"7368f6a9-46be-40c5-965b-78f2e3cfa301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"
Sep 11 00:02:17.476397 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.476359    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7368f6a9-46be-40c5-965b-78f2e3cfa301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j\" (UID: \"7368f6a9-46be-40c5-965b-78f2e3cfa301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"
Sep 11 00:02:17.477200 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.477173    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7368f6a9-46be-40c5-965b-78f2e3cfa301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j\" (UID: \"7368f6a9-46be-40c5-965b-78f2e3cfa301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"
Sep 11 00:02:17.598824 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.598731    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"
Sep 11 00:02:17.869609 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:17.869579    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"]
Sep 11 00:02:17.871594 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:02:17.871568    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7368f6a9_46be_40c5_965b_78f2e3cfa301.slice/crio-053bf7ce8699698d595c38d2c025bb38a038950f72446bbd9aec406a57295656 WatchSource:0}: Error finding container 053bf7ce8699698d595c38d2c025bb38a038950f72446bbd9aec406a57295656: Status 404 returned error can't find the container with id 053bf7ce8699698d595c38d2c025bb38a038950f72446bbd9aec406a57295656
Sep 11 00:02:18.586643 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:18.586612    2569 generic.go:356] "Generic (PLEG): container finished" podID="7368f6a9-46be-40c5-965b-78f2e3cfa301" containerID="31cda5d96b365a2c6260abeb929251f818786bf389d3ed3958df4fb186b5e25d" exitCode=0
Sep 11 00:02:18.587038 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:18.586667    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j" event={"ID":"7368f6a9-46be-40c5-965b-78f2e3cfa301","Type":"ContainerDied","Data":"31cda5d96b365a2c6260abeb929251f818786bf389d3ed3958df4fb186b5e25d"}
Sep 11 00:02:18.587038 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:18.586710    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j" event={"ID":"7368f6a9-46be-40c5-965b-78f2e3cfa301","Type":"ContainerStarted","Data":"053bf7ce8699698d595c38d2c025bb38a038950f72446bbd9aec406a57295656"}
Sep 11 00:02:19.732919 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:19.732893    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"
Sep 11 00:02:19.796412 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:19.796382    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7368f6a9-46be-40c5-965b-78f2e3cfa301-maas-api-service-ca\") pod \"7368f6a9-46be-40c5-965b-78f2e3cfa301\" (UID: \"7368f6a9-46be-40c5-965b-78f2e3cfa301\") "
Sep 11 00:02:19.796740 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:19.796719    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7368f6a9-46be-40c5-965b-78f2e3cfa301-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7368f6a9-46be-40c5-965b-78f2e3cfa301" (UID: "7368f6a9-46be-40c5-965b-78f2e3cfa301"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:19.897493 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:19.897464    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7368f6a9-46be-40c5-965b-78f2e3cfa301-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:20.598243 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:20.598211    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"
Sep 11 00:02:20.598422 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:20.598251    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j" event={"ID":"7368f6a9-46be-40c5-965b-78f2e3cfa301","Type":"ContainerDied","Data":"053bf7ce8699698d595c38d2c025bb38a038950f72446bbd9aec406a57295656"}
Sep 11 00:02:20.598422 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:20.598300    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="053bf7ce8699698d595c38d2c025bb38a038950f72446bbd9aec406a57295656"
Sep 11 00:02:27.753480 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:27.753446    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"]
Sep 11 00:02:27.753895 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:27.753681    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d" podUID="e3454cdc-1a6c-4420-8191-50d59c65e336" containerName="payload-processing" containerID="cri-o://268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90" gracePeriod=30
Sep 11 00:02:27.773288 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:27.773258    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"]
Sep 11 00:02:27.773509 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:27.773456    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf" podUID="60f741ba-7190-43c8-bec7-fcfd06ac79d6" containerName="payload-pre-processing" containerID="cri-o://610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa" gracePeriod=30
Sep 11 00:02:27.934342 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:27.934298    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"]
Sep 11 00:02:27.936177 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:27.936152    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-0b637b-gmm9j"]
Sep 11 00:02:28.056434 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.056401    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 11 00:02:28.059095 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.059054    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 11 00:02:28.172284 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.172249    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60f741ba-7190-43c8-bec7-fcfd06ac79d6-plugins-config-volume\") pod \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\" (UID: \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\") "
Sep 11 00:02:28.172454 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.172310    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwtbd\" (UniqueName: \"kubernetes.io/projected/60f741ba-7190-43c8-bec7-fcfd06ac79d6-kube-api-access-kwtbd\") pod \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\" (UID: \"60f741ba-7190-43c8-bec7-fcfd06ac79d6\") "
Sep 11 00:02:28.172454 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.172359    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7dbt\" (UniqueName: \"kubernetes.io/projected/e3454cdc-1a6c-4420-8191-50d59c65e336-kube-api-access-w7dbt\") pod \"e3454cdc-1a6c-4420-8191-50d59c65e336\" (UID: \"e3454cdc-1a6c-4420-8191-50d59c65e336\") "
Sep 11 00:02:28.172454 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.172386    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3454cdc-1a6c-4420-8191-50d59c65e336-plugins-config-volume\") pod \"e3454cdc-1a6c-4420-8191-50d59c65e336\" (UID: \"e3454cdc-1a6c-4420-8191-50d59c65e336\") "
Sep 11 00:02:28.172675 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.172648    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60f741ba-7190-43c8-bec7-fcfd06ac79d6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "60f741ba-7190-43c8-bec7-fcfd06ac79d6" (UID: "60f741ba-7190-43c8-bec7-fcfd06ac79d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:28.172845 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.172826    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3454cdc-1a6c-4420-8191-50d59c65e336-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3454cdc-1a6c-4420-8191-50d59c65e336" (UID: "e3454cdc-1a6c-4420-8191-50d59c65e336"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:28.174480 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.174446    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3454cdc-1a6c-4420-8191-50d59c65e336-kube-api-access-w7dbt" (OuterVolumeSpecName: "kube-api-access-w7dbt") pod "e3454cdc-1a6c-4420-8191-50d59c65e336" (UID: "e3454cdc-1a6c-4420-8191-50d59c65e336"). InnerVolumeSpecName "kube-api-access-w7dbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:28.174582 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.174504    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60f741ba-7190-43c8-bec7-fcfd06ac79d6-kube-api-access-kwtbd" (OuterVolumeSpecName: "kube-api-access-kwtbd") pod "60f741ba-7190-43c8-bec7-fcfd06ac79d6" (UID: "60f741ba-7190-43c8-bec7-fcfd06ac79d6"). InnerVolumeSpecName "kube-api-access-kwtbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:28.273902 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.273865    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3454cdc-1a6c-4420-8191-50d59c65e336-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:28.273902 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.273896    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60f741ba-7190-43c8-bec7-fcfd06ac79d6-plugins-config-volume\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:28.273902 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.273906    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwtbd\" (UniqueName: \"kubernetes.io/projected/60f741ba-7190-43c8-bec7-fcfd06ac79d6-kube-api-access-kwtbd\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:28.274175 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.273914    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7dbt\" (UniqueName: \"kubernetes.io/projected/e3454cdc-1a6c-4420-8191-50d59c65e336-kube-api-access-w7dbt\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:28.641227 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.641192    2569 generic.go:356] "Generic (PLEG): container finished" podID="e3454cdc-1a6c-4420-8191-50d59c65e336" containerID="268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90" exitCode=0
Sep 11 00:02:28.641418 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.641253    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"
Sep 11 00:02:28.641418 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.641282    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d" event={"ID":"e3454cdc-1a6c-4420-8191-50d59c65e336","Type":"ContainerDied","Data":"268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90"}
Sep 11 00:02:28.641418 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.641342    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d" event={"ID":"e3454cdc-1a6c-4420-8191-50d59c65e336","Type":"ContainerDied","Data":"142a16ba58f50ccdc51e19c494cbe169540e447dddfe1885167afdee2e57294e"}
Sep 11 00:02:28.641418 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.641370    2569 scope.go:117] "RemoveContainer" containerID="268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90"
Sep 11 00:02:28.642684 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.642661    2569 generic.go:356] "Generic (PLEG): container finished" podID="60f741ba-7190-43c8-bec7-fcfd06ac79d6" containerID="610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa" exitCode=0
Sep 11 00:02:28.642772 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.642715    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"
Sep 11 00:02:28.642772 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.642751    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf" event={"ID":"60f741ba-7190-43c8-bec7-fcfd06ac79d6","Type":"ContainerDied","Data":"610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa"}
Sep 11 00:02:28.642851 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.642795    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf" event={"ID":"60f741ba-7190-43c8-bec7-fcfd06ac79d6","Type":"ContainerDied","Data":"f01a461f2b9dbf89d2bf7f27bd0d85a9637894abcfc5c0a20fca45932700368e"}
Sep 11 00:02:28.652203 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.652183    2569 scope.go:117] "RemoveContainer" containerID="268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90"
Sep 11 00:02:28.652516 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:02:28.652490    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90\": container with ID starting with 268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90 not found: ID does not exist" containerID="268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90"
Sep 11 00:02:28.652608 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.652528    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90"} err="failed to get container status \"268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90\": rpc error: code = NotFound desc = could not find container \"268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90\": container with ID starting with 268b2fbce33ab633da735722b5ec9bd9a300bafc1b90370f6becc0c9f1effa90 not found: ID does not exist"
Sep 11 00:02:28.652608 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.652552    2569 scope.go:117] "RemoveContainer" containerID="610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa"
Sep 11 00:02:28.660985 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.660967    2569 scope.go:117] "RemoveContainer" containerID="610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa"
Sep 11 00:02:28.661264 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:02:28.661246    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa\": container with ID starting with 610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa not found: ID does not exist" containerID="610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa"
Sep 11 00:02:28.661331 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.661270    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa"} err="failed to get container status \"610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa\": rpc error: code = NotFound desc = could not find container \"610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa\": container with ID starting with 610f88d3ca6a6d317f8d98aea45389a8c081d3865cff413ed2ce28c0544a97fa not found: ID does not exist"
Sep 11 00:02:28.662133 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.662113    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"]
Sep 11 00:02:28.665901 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.665880    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-0b637b-548fb948f9-cws4d"]
Sep 11 00:02:28.673731 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.673710    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"]
Sep 11 00:02:28.676867 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.676846    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0b637b-59567f4554-85zrf"]
Sep 11 00:02:28.979006 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.978906    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60f741ba-7190-43c8-bec7-fcfd06ac79d6" path="/var/lib/kubelet/pods/60f741ba-7190-43c8-bec7-fcfd06ac79d6/volumes"
Sep 11 00:02:28.979347 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.979262    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7368f6a9-46be-40c5-965b-78f2e3cfa301" path="/var/lib/kubelet/pods/7368f6a9-46be-40c5-965b-78f2e3cfa301/volumes"
Sep 11 00:02:28.979601 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:28.979587    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3454cdc-1a6c-4420-8191-50d59c65e336" path="/var/lib/kubelet/pods/e3454cdc-1a6c-4420-8191-50d59c65e336/volumes"
Sep 11 00:02:34.693024 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:34.692984    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"]
Sep 11 00:02:34.693467 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:34.693243    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy" containerID="cri-o://d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3" gracePeriod=30
Sep 11 00:02:35.039891 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.039809    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"]
Sep 11 00:02:35.040206 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040191    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60f741ba-7190-43c8-bec7-fcfd06ac79d6" containerName="payload-pre-processing"
Sep 11 00:02:35.040255 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040207    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f741ba-7190-43c8-bec7-fcfd06ac79d6" containerName="payload-pre-processing"
Sep 11 00:02:35.040255 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040218    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7368f6a9-46be-40c5-965b-78f2e3cfa301" containerName="revoke-keys"
Sep 11 00:02:35.040255 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040223    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7368f6a9-46be-40c5-965b-78f2e3cfa301" containerName="revoke-keys"
Sep 11 00:02:35.040255 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040243    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3454cdc-1a6c-4420-8191-50d59c65e336" containerName="payload-processing"
Sep 11 00:02:35.040255 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040249    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3454cdc-1a6c-4420-8191-50d59c65e336" containerName="payload-processing"
Sep 11 00:02:35.040433 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040301    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7368f6a9-46be-40c5-965b-78f2e3cfa301" containerName="revoke-keys"
Sep 11 00:02:35.040433 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040308    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3454cdc-1a6c-4420-8191-50d59c65e336" containerName="payload-processing"
Sep 11 00:02:35.040433 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040316    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0d9fb1-8d59-4f6b-bc88-6097657cb5bc" containerName="istio-proxy"
Sep 11 00:02:35.040433 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.040324    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="60f741ba-7190-43c8-bec7-fcfd06ac79d6" containerName="payload-pre-processing"
Sep 11 00:02:35.045126 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.045106    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"
Sep 11 00:02:35.047341 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.047321    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 00:02:35.047438 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.047384    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-55278\""
Sep 11 00:02:35.055110 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.055087    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"]
Sep 11 00:02:35.136768 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.136734    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/756fbe62-c695-4691-a0c6-39279bf2ce4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv\" (UID: \"756fbe62-c695-4691-a0c6-39279bf2ce4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"
Sep 11 00:02:35.238250 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.238213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/756fbe62-c695-4691-a0c6-39279bf2ce4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv\" (UID: \"756fbe62-c695-4691-a0c6-39279bf2ce4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"
Sep 11 00:02:35.238841 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.238810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/756fbe62-c695-4691-a0c6-39279bf2ce4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv\" (UID: \"756fbe62-c695-4691-a0c6-39279bf2ce4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"
Sep 11 00:02:35.356140 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.356053    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"
Sep 11 00:02:35.630710 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.630686    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"]
Sep 11 00:02:35.632818 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:02:35.632794    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756fbe62_c695_4691_a0c6_39279bf2ce4e.slice/crio-a1d948cb1018afd0a08f953fa3357f18e4f62d4e4fad0ce089b0f6a0b28bf561 WatchSource:0}: Error finding container a1d948cb1018afd0a08f953fa3357f18e4f62d4e4fad0ce089b0f6a0b28bf561: Status 404 returned error can't find the container with id a1d948cb1018afd0a08f953fa3357f18e4f62d4e4fad0ce089b0f6a0b28bf561
Sep 11 00:02:35.682851 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:35.682822    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv" event={"ID":"756fbe62-c695-4691-a0c6-39279bf2ce4e","Type":"ContainerStarted","Data":"a1d948cb1018afd0a08f953fa3357f18e4f62d4e4fad0ce089b0f6a0b28bf561"}
Sep 11 00:02:36.690226 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:36.690191    2569 generic.go:356] "Generic (PLEG): container finished" podID="756fbe62-c695-4691-a0c6-39279bf2ce4e" containerID="fa03c9f76d1af0e59fe89df356ae63455292fd821fc02fe34ab9ec239c4ebb9c" exitCode=0
Sep 11 00:02:36.690595 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:36.690233    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv" event={"ID":"756fbe62-c695-4691-a0c6-39279bf2ce4e","Type":"ContainerDied","Data":"fa03c9f76d1af0e59fe89df356ae63455292fd821fc02fe34ab9ec239c4ebb9c"}
Sep 11 00:02:36.925152 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:36.925114    2569 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 00:02:36.925308 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:36.925174    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 00:02:37.830852 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:37.830824    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"
Sep 11 00:02:37.964703 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:37.964608    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/756fbe62-c695-4691-a0c6-39279bf2ce4e-maas-api-service-ca\") pod \"756fbe62-c695-4691-a0c6-39279bf2ce4e\" (UID: \"756fbe62-c695-4691-a0c6-39279bf2ce4e\") "
Sep 11 00:02:37.965024 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:37.964993    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/756fbe62-c695-4691-a0c6-39279bf2ce4e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "756fbe62-c695-4691-a0c6-39279bf2ce4e" (UID: "756fbe62-c695-4691-a0c6-39279bf2ce4e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:38.065697 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:38.065646    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/756fbe62-c695-4691-a0c6-39279bf2ce4e-maas-api-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:38.701822 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:38.701791    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"
Sep 11 00:02:38.701822 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:38.701805    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv" event={"ID":"756fbe62-c695-4691-a0c6-39279bf2ce4e","Type":"ContainerDied","Data":"a1d948cb1018afd0a08f953fa3357f18e4f62d4e4fad0ce089b0f6a0b28bf561"}
Sep 11 00:02:38.702070 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:38.701851    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1d948cb1018afd0a08f953fa3357f18e4f62d4e4fad0ce089b0f6a0b28bf561"
Sep 11 00:02:39.951577 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:39.951552    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 11 00:02:40.085497 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085398    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-token\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.085497 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085453    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-credential-socket\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.085722 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085502    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-podinfo\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.085722 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085519    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqjwn\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-kube-api-access-fqjwn\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.085722 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085544    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-socket\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.085722 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085570    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-certs\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.085722 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085636    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-data\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.085722 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085698    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-envoy\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.085722 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085722    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istiod-ca-cert\") pod \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\" (UID: \"132ee4ff-a5ea-4897-a2e2-c246dbb7743c\") "
Sep 11 00:02:40.086133 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085918    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:40.086133 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085954    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:40.086133 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.085967    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:40.086133 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.086099    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-data" (OuterVolumeSpecName: "istio-data") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:40.086340 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.086163    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.086340 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.086185    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.086340 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.086202    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.086340 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.086267    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:40.087704 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.087678    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-token" (OuterVolumeSpecName: "istio-token") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:40.087842 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.087753    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 00:02:40.088177 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.088159    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:40.088681 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.088659    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-kube-api-access-fqjwn" (OuterVolumeSpecName: "kube-api-access-fqjwn") pod "132ee4ff-a5ea-4897-a2e2-c246dbb7743c" (UID: "132ee4ff-a5ea-4897-a2e2-c246dbb7743c"). InnerVolumeSpecName "kube-api-access-fqjwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:40.187169 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.187123    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.187169 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.187165    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqjwn\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-kube-api-access-fqjwn\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.187169 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.187176    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.187169 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.187187    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.187445 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.187195    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.187445 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.187203    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/132ee4ff-a5ea-4897-a2e2-c246dbb7743c-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:40.715451 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.715413    2569 generic.go:356] "Generic (PLEG): container finished" podID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerID="d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3" exitCode=0
Sep 11 00:02:40.715632 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.715458    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"
Sep 11 00:02:40.715632 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.715497    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" event={"ID":"132ee4ff-a5ea-4897-a2e2-c246dbb7743c","Type":"ContainerDied","Data":"d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3"}
Sep 11 00:02:40.715632 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.715546    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb" event={"ID":"132ee4ff-a5ea-4897-a2e2-c246dbb7743c","Type":"ContainerDied","Data":"eeace63a8447515207d0d28a531effbdba14ebeca13f208aaf399aff1016b629"}
Sep 11 00:02:40.715632 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.715568    2569 scope.go:117] "RemoveContainer" containerID="d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3"
Sep 11 00:02:40.725290 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.725274    2569 scope.go:117] "RemoveContainer" containerID="218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a"
Sep 11 00:02:40.733615 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.733599    2569 scope.go:117] "RemoveContainer" containerID="d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3"
Sep 11 00:02:40.733879 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:02:40.733863    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3\": container with ID starting with d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3 not found: ID does not exist" containerID="d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3"
Sep 11 00:02:40.733944 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.733886    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3"} err="failed to get container status \"d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3\": rpc error: code = NotFound desc = could not find container \"d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3\": container with ID starting with d2e836134a12d300db3623dbd1f06bfbdb70b3e738156a19d8b242a7296ad0e3 not found: ID does not exist"
Sep 11 00:02:40.733944 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.733901    2569 scope.go:117] "RemoveContainer" containerID="218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a"
Sep 11 00:02:40.734143 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:02:40.734128    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a\": container with ID starting with 218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a not found: ID does not exist" containerID="218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a"
Sep 11 00:02:40.734185 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.734145    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a"} err="failed to get container status \"218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a\": rpc error: code = NotFound desc = could not find container \"218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a\": container with ID starting with 218753896e6ff319b79b54792ea18ba4647f0699f446726c09a5f7152e96e21a not found: ID does not exist"
Sep 11 00:02:40.737312 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.737292    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"]
Sep 11 00:02:40.741279 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.741257    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0b637b-openshift-default-65db6fd976-92hbb"]
Sep 11 00:02:40.978032 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:40.977926    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" path="/var/lib/kubelet/pods/132ee4ff-a5ea-4897-a2e2-c246dbb7743c/volumes"
Sep 11 00:02:45.521836 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:45.521802    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"]
Sep 11 00:02:45.523680 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:45.523651    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-6d2e31-29818080-vnnp2"]
Sep 11 00:02:45.710532 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:45.710498    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"]
Sep 11 00:02:45.714622 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:45.714597    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-6d2e31-hmmxv"]
Sep 11 00:02:46.977966 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:46.977916    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63019e8c-dfdd-48d7-8383-e2d3593e46ba" path="/var/lib/kubelet/pods/63019e8c-dfdd-48d7-8383-e2d3593e46ba/volumes"
Sep 11 00:02:46.978339 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:46.978245    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="756fbe62-c695-4691-a0c6-39279bf2ce4e" path="/var/lib/kubelet/pods/756fbe62-c695-4691-a0c6-39279bf2ce4e/volumes"
Sep 11 00:02:52.297522 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:52.297480    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"]
Sep 11 00:02:52.298131 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:52.297725    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy" containerID="cri-o://46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f" gracePeriod=30
Sep 11 00:02:57.555261 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.555233    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 11 00:02:57.647997 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.647966    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-data\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648008    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-envoy\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648024    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-podinfo\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648051    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4cw7\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-kube-api-access-k4cw7\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648073    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-token\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648095    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-certs\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648172    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41546028-449d-41b6-bb75-0e6b3400bdbf-istiod-ca-cert\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648196    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-socket\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648602 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648261    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-credential-socket\") pod \"41546028-449d-41b6-bb75-0e6b3400bdbf\" (UID: \"41546028-449d-41b6-bb75-0e6b3400bdbf\") "
Sep 11 00:02:57.648602 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648374    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-data" (OuterVolumeSpecName: "istio-data") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:57.648602 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648521    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:57.648602 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648563    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:57.648811 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648627    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41546028-449d-41b6-bb75-0e6b3400bdbf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:02:57.648811 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648670    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.648921 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648808    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-credential-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.648921 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648826    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-data\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.648921 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.648756    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:57.650411 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.650382    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 00:02:57.650521 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.650448    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:02:57.650596 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.650581    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-token" (OuterVolumeSpecName: "istio-token") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:57.650769 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.650755    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-kube-api-access-k4cw7" (OuterVolumeSpecName: "kube-api-access-k4cw7") pod "41546028-449d-41b6-bb75-0e6b3400bdbf" (UID: "41546028-449d-41b6-bb75-0e6b3400bdbf"). InnerVolumeSpecName "kube-api-access-k4cw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:02:57.749723 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.749670    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41546028-449d-41b6-bb75-0e6b3400bdbf-istiod-ca-cert\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.749723 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.749715    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-workload-socket\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.749723 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.749725    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-envoy\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.749723 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.749733    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-podinfo\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.750030 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.749743    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4cw7\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-kube-api-access-k4cw7\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.750030 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.749752    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41546028-449d-41b6-bb75-0e6b3400bdbf-istio-token\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:02:57.811135 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.811095    2569 generic.go:356] "Generic (PLEG): container finished" podID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerID="46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f" exitCode=0
Sep 11 00:02:57.811321 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.811139    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"
Sep 11 00:02:57.811321 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.811171    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" event={"ID":"41546028-449d-41b6-bb75-0e6b3400bdbf","Type":"ContainerDied","Data":"46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f"}
Sep 11 00:02:57.811321 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.811271    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h" event={"ID":"41546028-449d-41b6-bb75-0e6b3400bdbf","Type":"ContainerDied","Data":"2335413a377d144e5b2ed8aa4835ba6431df059c03a9f2ad46825b130ae59b47"}
Sep 11 00:02:57.811321 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.811306    2569 scope.go:117] "RemoveContainer" containerID="46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f"
Sep 11 00:02:57.825102 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.825069    2569 scope.go:117] "RemoveContainer" containerID="675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1"
Sep 11 00:02:57.835739 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.835716    2569 scope.go:117] "RemoveContainer" containerID="46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f"
Sep 11 00:02:57.835839 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.835809    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"]
Sep 11 00:02:57.836101 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:02:57.836080    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f\": container with ID starting with 46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f not found: ID does not exist" containerID="46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f"
Sep 11 00:02:57.836202 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.836110    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f"} err="failed to get container status \"46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f\": rpc error: code = NotFound desc = could not find container \"46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f\": container with ID starting with 46f5bc8b6532020faee90e8398d11c6e172cfe0df4edae0ce1676083ae4c412f not found: ID does not exist"
Sep 11 00:02:57.836202 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.836128    2569 scope.go:117] "RemoveContainer" containerID="675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1"
Sep 11 00:02:57.836452 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:02:57.836422    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1\": container with ID starting with 675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1 not found: ID does not exist" containerID="675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1"
Sep 11 00:02:57.836524 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.836463    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1"} err="failed to get container status \"675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1\": rpc error: code = NotFound desc = could not find container \"675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1\": container with ID starting with 675e8ff7816480e5b70f9fc5b92a6036d2924540e7827eacf34933ae314233b1 not found: ID does not exist"
Sep 11 00:02:57.840881 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:57.840854    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-6d2e31-openshift-default-57bccdbc56-ct94h"]
Sep 11 00:02:58.978804 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:02:58.978770    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" path="/var/lib/kubelet/pods/41546028-449d-41b6-bb75-0e6b3400bdbf/volumes"
Sep 11 00:03:07.133743 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:03:07.133714    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 11 00:03:07.135258 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:03:07.135235    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 11 00:03:07.139274 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:03:07.139259    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 11 00:03:07.140869 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:03:07.140853    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 11 00:03:07.390145 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:03:07.390065    2569 scope.go:117] "RemoveContainer" containerID="edccef9406b6476596646dcd6a4e6742327e21cad205d9f0a4d3d637f7ebb67d"
Sep 11 00:03:07.398797 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:03:07.398776    2569 scope.go:117] "RemoveContainer" containerID="0305e9887649663a6619ff0789a118015e6af8ee1d03a79c2b21bdbf26802a25"
Sep 11 00:04:07.446263 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:04:07.446230    2569 scope.go:117] "RemoveContainer" containerID="56b03edfbdc9e19972a73af4f5261c7a059fcc46cbf9c70e5bd87147e32287bb"
Sep 11 00:06:07.472987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:06:07.472926    2569 scope.go:117] "RemoveContainer" containerID="66f428a6ed5e55121ac15d5728ecf266853ed46ed1dd51b3a3d833c759faf921"
Sep 11 00:07:07.492085 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:07.492053    2569 scope.go:117] "RemoveContainer" containerID="3b2a79dd2de0b01dbec1ab4d4f5ae00e59bb56cd6c890ff9c0905047dd21e60e"
Sep 11 00:07:20.904180 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904141    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"]
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904483    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy"
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904495    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy"
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904509    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy"
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904514    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy"
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904533    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="756fbe62-c695-4691-a0c6-39279bf2ce4e" containerName="revoke-keys"
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904539    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="756fbe62-c695-4691-a0c6-39279bf2ce4e" containerName="revoke-keys"
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904588    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy"
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904597    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="756fbe62-c695-4691-a0c6-39279bf2ce4e" containerName="revoke-keys"
Sep 11 00:07:20.904625 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904613    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy"
Sep 11 00:07:20.904910 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904622    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy"
Sep 11 00:07:20.904910 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.904844    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy"
Sep 11 00:07:20.907869 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.907853    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:20.910667 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.910637    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-13dc3fb1-kserve-self-signed-certs\""
Sep 11 00:07:20.912513 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.912491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-tmp-dir\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:20.912642 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.912540    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-home\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:20.912642 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.912576    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-dshm\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:20.912642 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.912617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-model-cache\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:20.912757 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.912683    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/740c1955-5904-491d-a27c-3e51e803b68f-tls-certs\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:20.912792 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.912761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfbt8\" (UniqueName: \"kubernetes.io/projected/740c1955-5904-491d-a27c-3e51e803b68f-kube-api-access-nfbt8\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:20.916312 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:20.916285    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"]
Sep 11 00:07:21.018473 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.018434    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-tmp-dir\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.018673 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.018485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-home\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.018673 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.018518    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-dshm\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.018673 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.018552    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-model-cache\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.018673 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.018604    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/740c1955-5904-491d-a27c-3e51e803b68f-tls-certs\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.018673 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.018637    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfbt8\" (UniqueName: \"kubernetes.io/projected/740c1955-5904-491d-a27c-3e51e803b68f-kube-api-access-nfbt8\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.018988 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.018960    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-tmp-dir\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.019118 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.018997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-home\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.019118 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.019059    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-model-cache\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.020945 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.020911    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-dshm\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.021091 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.021072    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/740c1955-5904-491d-a27c-3e51e803b68f-tls-certs\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.025219 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.025202    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfbt8\" (UniqueName: \"kubernetes.io/projected/740c1955-5904-491d-a27c-3e51e803b68f-kube-api-access-nfbt8\") pod \"e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") " pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.218738 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.218647    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:21.489495 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:21.489470    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"]
Sep 11 00:07:21.492193 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:07:21.492161    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod740c1955_5904_491d_a27c_3e51e803b68f.slice/crio-ee1bfbeb1471125a8c22a54eff32f429013a610acb811828e32dd47103ed796b WatchSource:0}: Error finding container ee1bfbeb1471125a8c22a54eff32f429013a610acb811828e32dd47103ed796b: Status 404 returned error can't find the container with id ee1bfbeb1471125a8c22a54eff32f429013a610acb811828e32dd47103ed796b
Sep 11 00:07:22.115159 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:22.115120    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl" event={"ID":"740c1955-5904-491d-a27c-3e51e803b68f","Type":"ContainerStarted","Data":"17737ca2798b5410e64f046185554a83d7b07c8909b0feb34d18ffdd0b811369"}
Sep 11 00:07:22.115159 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:22.115165    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl" event={"ID":"740c1955-5904-491d-a27c-3e51e803b68f","Type":"ContainerStarted","Data":"ee1bfbeb1471125a8c22a54eff32f429013a610acb811828e32dd47103ed796b"}
Sep 11 00:07:22.130018 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:22.129970    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl" podStartSLOduration=2.12991571 podStartE2EDuration="2.12991571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:07:22.128491332 +0000 UTC m=+2055.782211506" watchObservedRunningTime="2026-09-11 00:07:22.12991571 +0000 UTC m=+2055.783635883"
Sep 11 00:07:24.786811 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.786769    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"]
Sep 11 00:07:24.787419 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.787397    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy"
Sep 11 00:07:24.787464 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.787423    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="132ee4ff-a5ea-4897-a2e2-c246dbb7743c" containerName="istio-proxy"
Sep 11 00:07:24.787464 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.787451    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy"
Sep 11 00:07:24.787464 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.787460    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="41546028-449d-41b6-bb75-0e6b3400bdbf" containerName="istio-proxy"
Sep 11 00:07:24.791035 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.791014    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.793075 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.793054    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-13dc3fb1-kserve-self-signed-certs\""
Sep 11 00:07:24.796752 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.796729    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"]
Sep 11 00:07:24.855257 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.855218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-model-cache\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.855257 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.855272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-home\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.855552 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.855289    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tls-certs\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.855552 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.855383    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ftwn\" (UniqueName: \"kubernetes.io/projected/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-kube-api-access-9ftwn\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.855552 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.855509    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-dshm\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.855682 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.855556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tmp-dir\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956257 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-dshm\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956257 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tmp-dir\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956529 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-model-cache\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956529 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-home\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956529 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956353    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tls-certs\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956529 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956396    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ftwn\" (UniqueName: \"kubernetes.io/projected/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-kube-api-access-9ftwn\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956751 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956724    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-model-cache\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956817 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956766    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tmp-dir\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.956927 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.956908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-home\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.958730 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.958700    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-dshm\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.958987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.958971    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tls-certs\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:24.966328 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:24.966305    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ftwn\" (UniqueName: \"kubernetes.io/projected/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-kube-api-access-9ftwn\") pod \"e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") " pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:25.103688 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:25.103597    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:25.383748 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:25.383713    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"]
Sep 11 00:07:25.387171 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:07:25.387140    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95c53c5e_bb4d_4e02_a780_3262fc2e6da8.slice/crio-b75c99e56a53a33dc370ce1c7263eac8774c5f0d4f5796dfa0046a7c4938482c WatchSource:0}: Error finding container b75c99e56a53a33dc370ce1c7263eac8774c5f0d4f5796dfa0046a7c4938482c: Status 404 returned error can't find the container with id b75c99e56a53a33dc370ce1c7263eac8774c5f0d4f5796dfa0046a7c4938482c
Sep 11 00:07:26.137233 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:26.137169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5" event={"ID":"95c53c5e-bb4d-4e02-a780-3262fc2e6da8","Type":"ContainerStarted","Data":"add52e72fd862cd4130a9cdc6426624623134d85c3b3b885c1e061d6bf726cd0"}
Sep 11 00:07:26.137233 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:26.137232    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5" event={"ID":"95c53c5e-bb4d-4e02-a780-3262fc2e6da8","Type":"ContainerStarted","Data":"b75c99e56a53a33dc370ce1c7263eac8774c5f0d4f5796dfa0046a7c4938482c"}
Sep 11 00:07:26.152756 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:26.152684    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5" podStartSLOduration=2.152668324 podStartE2EDuration="2.152668324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:07:26.151907113 +0000 UTC m=+2059.805627285" watchObservedRunningTime="2026-09-11 00:07:26.152668324 +0000 UTC m=+2059.806388496"
Sep 11 00:07:31.219819 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:31.219739    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:31.220223 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:31.219952    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:31.232222 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:31.232195    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:32.180833 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:32.180799    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:35.104627 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:35.104580    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:35.104627 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:35.104629    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:35.117108 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:35.117082    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:35.195400 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:35.195371    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:07:44.068384 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:44.068351    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"]
Sep 11 00:07:45.079660 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:45.079621    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"]
Sep 11 00:07:59.091314 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.091273    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl" podUID="740c1955-5904-491d-a27c-3e51e803b68f" containerName="main" containerID="cri-o://17737ca2798b5410e64f046185554a83d7b07c8909b0feb34d18ffdd0b811369" gracePeriod=45
Sep 11 00:07:59.301218 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.301185    2569 generic.go:356] "Generic (PLEG): container finished" podID="740c1955-5904-491d-a27c-3e51e803b68f" containerID="17737ca2798b5410e64f046185554a83d7b07c8909b0feb34d18ffdd0b811369" exitCode=0
Sep 11 00:07:59.301396 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.301253    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl" event={"ID":"740c1955-5904-491d-a27c-3e51e803b68f","Type":"ContainerDied","Data":"17737ca2798b5410e64f046185554a83d7b07c8909b0feb34d18ffdd0b811369"}
Sep 11 00:07:59.347847 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.347787    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:07:59.391386 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391345    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/740c1955-5904-491d-a27c-3e51e803b68f-tls-certs\") pod \"740c1955-5904-491d-a27c-3e51e803b68f\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") "
Sep 11 00:07:59.391386 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391391    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-tmp-dir\") pod \"740c1955-5904-491d-a27c-3e51e803b68f\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") "
Sep 11 00:07:59.391649 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391415    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-model-cache\") pod \"740c1955-5904-491d-a27c-3e51e803b68f\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") "
Sep 11 00:07:59.391649 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391437    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-dshm\") pod \"740c1955-5904-491d-a27c-3e51e803b68f\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") "
Sep 11 00:07:59.391649 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391469    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-home\") pod \"740c1955-5904-491d-a27c-3e51e803b68f\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") "
Sep 11 00:07:59.391649 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391505    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfbt8\" (UniqueName: \"kubernetes.io/projected/740c1955-5904-491d-a27c-3e51e803b68f-kube-api-access-nfbt8\") pod \"740c1955-5904-491d-a27c-3e51e803b68f\" (UID: \"740c1955-5904-491d-a27c-3e51e803b68f\") "
Sep 11 00:07:59.391872 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391670    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "740c1955-5904-491d-a27c-3e51e803b68f" (UID: "740c1955-5904-491d-a27c-3e51e803b68f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:07:59.391872 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391681    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-model-cache" (OuterVolumeSpecName: "model-cache") pod "740c1955-5904-491d-a27c-3e51e803b68f" (UID: "740c1955-5904-491d-a27c-3e51e803b68f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:07:59.391872 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391720    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-home" (OuterVolumeSpecName: "home") pod "740c1955-5904-491d-a27c-3e51e803b68f" (UID: "740c1955-5904-491d-a27c-3e51e803b68f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:07:59.391872 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391841    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-tmp-dir\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:07:59.391872 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391867    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-model-cache\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:07:59.392175 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.391882    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-home\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:07:59.393700 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.393675    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/740c1955-5904-491d-a27c-3e51e803b68f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "740c1955-5904-491d-a27c-3e51e803b68f" (UID: "740c1955-5904-491d-a27c-3e51e803b68f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:07:59.393831 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.393741    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/740c1955-5904-491d-a27c-3e51e803b68f-kube-api-access-nfbt8" (OuterVolumeSpecName: "kube-api-access-nfbt8") pod "740c1955-5904-491d-a27c-3e51e803b68f" (UID: "740c1955-5904-491d-a27c-3e51e803b68f"). InnerVolumeSpecName "kube-api-access-nfbt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:07:59.393831 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.393779    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-dshm" (OuterVolumeSpecName: "dshm") pod "740c1955-5904-491d-a27c-3e51e803b68f" (UID: "740c1955-5904-491d-a27c-3e51e803b68f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:07:59.493046 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.493009    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/740c1955-5904-491d-a27c-3e51e803b68f-dshm\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:07:59.493046 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.493038    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfbt8\" (UniqueName: \"kubernetes.io/projected/740c1955-5904-491d-a27c-3e51e803b68f-kube-api-access-nfbt8\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:07:59.493046 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:07:59.493049    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/740c1955-5904-491d-a27c-3e51e803b68f-tls-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.099557 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.099501    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5" podUID="95c53c5e-bb4d-4e02-a780-3262fc2e6da8" containerName="main" containerID="cri-o://add52e72fd862cd4130a9cdc6426624623134d85c3b3b885c1e061d6bf726cd0" gracePeriod=45
Sep 11 00:08:00.316678 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.316642    2569 generic.go:356] "Generic (PLEG): container finished" podID="95c53c5e-bb4d-4e02-a780-3262fc2e6da8" containerID="add52e72fd862cd4130a9cdc6426624623134d85c3b3b885c1e061d6bf726cd0" exitCode=0
Sep 11 00:08:00.316833 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.316716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5" event={"ID":"95c53c5e-bb4d-4e02-a780-3262fc2e6da8","Type":"ContainerDied","Data":"add52e72fd862cd4130a9cdc6426624623134d85c3b3b885c1e061d6bf726cd0"}
Sep 11 00:08:00.318109 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.318087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl" event={"ID":"740c1955-5904-491d-a27c-3e51e803b68f","Type":"ContainerDied","Data":"ee1bfbeb1471125a8c22a54eff32f429013a610acb811828e32dd47103ed796b"}
Sep 11 00:08:00.318265 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.318125    2569 scope.go:117] "RemoveContainer" containerID="17737ca2798b5410e64f046185554a83d7b07c8909b0feb34d18ffdd0b811369"
Sep 11 00:08:00.318265 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.318124    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"
Sep 11 00:08:00.338483 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.338462    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"]
Sep 11 00:08:00.345213 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.345189    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-13dc3fb1-kserve-6759b79b75-shmcl"]
Sep 11 00:08:00.365287 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.365268    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:08:00.399773 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.399741    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-model-cache\") pod \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") "
Sep 11 00:08:00.399987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.399788    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tls-certs\") pod \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") "
Sep 11 00:08:00.399987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.399818    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tmp-dir\") pod \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") "
Sep 11 00:08:00.399987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.399851    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-dshm\") pod \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") "
Sep 11 00:08:00.399987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.399913    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-home\") pod \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") "
Sep 11 00:08:00.399987 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.399972    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ftwn\" (UniqueName: \"kubernetes.io/projected/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-kube-api-access-9ftwn\") pod \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\" (UID: \"95c53c5e-bb4d-4e02-a780-3262fc2e6da8\") "
Sep 11 00:08:00.400275 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.400044    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-model-cache" (OuterVolumeSpecName: "model-cache") pod "95c53c5e-bb4d-4e02-a780-3262fc2e6da8" (UID: "95c53c5e-bb4d-4e02-a780-3262fc2e6da8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:08:00.400275 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.400129    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "95c53c5e-bb4d-4e02-a780-3262fc2e6da8" (UID: "95c53c5e-bb4d-4e02-a780-3262fc2e6da8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:08:00.400382 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.400294    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-model-cache\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.400382 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.400337    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tmp-dir\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.400382 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.400333    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-home" (OuterVolumeSpecName: "home") pod "95c53c5e-bb4d-4e02-a780-3262fc2e6da8" (UID: "95c53c5e-bb4d-4e02-a780-3262fc2e6da8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:08:00.402115 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.402086    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "95c53c5e-bb4d-4e02-a780-3262fc2e6da8" (UID: "95c53c5e-bb4d-4e02-a780-3262fc2e6da8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:08:00.402221 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.402117    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-dshm" (OuterVolumeSpecName: "dshm") pod "95c53c5e-bb4d-4e02-a780-3262fc2e6da8" (UID: "95c53c5e-bb4d-4e02-a780-3262fc2e6da8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 00:08:00.402286 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.402252    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-kube-api-access-9ftwn" (OuterVolumeSpecName: "kube-api-access-9ftwn") pod "95c53c5e-bb4d-4e02-a780-3262fc2e6da8" (UID: "95c53c5e-bb4d-4e02-a780-3262fc2e6da8"). InnerVolumeSpecName "kube-api-access-9ftwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:08:00.501119 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.501083    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-home\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.501119 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.501114    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ftwn\" (UniqueName: \"kubernetes.io/projected/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-kube-api-access-9ftwn\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.501119 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.501126    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-tls-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.501343 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.501134    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95c53c5e-bb4d-4e02-a780-3262fc2e6da8-dshm\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:00.978900 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:00.978858    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="740c1955-5904-491d-a27c-3e51e803b68f" path="/var/lib/kubelet/pods/740c1955-5904-491d-a27c-3e51e803b68f/volumes"
Sep 11 00:08:01.325767 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:01.325688    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"
Sep 11 00:08:01.325767 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:01.325701    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5" event={"ID":"95c53c5e-bb4d-4e02-a780-3262fc2e6da8","Type":"ContainerDied","Data":"b75c99e56a53a33dc370ce1c7263eac8774c5f0d4f5796dfa0046a7c4938482c"}
Sep 11 00:08:01.325767 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:01.325743    2569 scope.go:117] "RemoveContainer" containerID="add52e72fd862cd4130a9cdc6426624623134d85c3b3b885c1e061d6bf726cd0"
Sep 11 00:08:01.339864 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:01.339841    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"]
Sep 11 00:08:01.343500 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:01.343477    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-13dc3fb1-kserve-6b755df5f9-7bbb5"]
Sep 11 00:08:02.001205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.001172    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"]
Sep 11 00:08:02.001554 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.001542    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c53c5e-bb4d-4e02-a780-3262fc2e6da8" containerName="main"
Sep 11 00:08:02.001608 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.001556    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c53c5e-bb4d-4e02-a780-3262fc2e6da8" containerName="main"
Sep 11 00:08:02.001608 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.001584    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="740c1955-5904-491d-a27c-3e51e803b68f" containerName="main"
Sep 11 00:08:02.001608 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.001590    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="740c1955-5904-491d-a27c-3e51e803b68f" containerName="main"
Sep 11 00:08:02.001701 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.001653    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="740c1955-5904-491d-a27c-3e51e803b68f" containerName="main"
Sep 11 00:08:02.001701 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.001663    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="95c53c5e-bb4d-4e02-a780-3262fc2e6da8" containerName="main"
Sep 11 00:08:02.006363 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.006341    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.009045 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.009017    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 00:08:02.009167 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.009042    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 00:08:02.009167 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.009112    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 00:08:02.009167 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.009114    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7fgh2\""
Sep 11 00:08:02.009167 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.009117    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 00:08:02.013795 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.013771    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911182b-3807-473b-b24a-59ee7610dd11-openshift-service-ca\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.013890 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.013818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-metrics-certs\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.013890 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.013858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-maas-api-tls\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.014582 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.013985    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq6zc\" (UniqueName: \"kubernetes.io/projected/f911182b-3807-473b-b24a-59ee7610dd11-kube-api-access-kq6zc\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.015496 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.015464    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"]
Sep 11 00:08:02.115164 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.115131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-maas-api-tls\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.115357 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.115222    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq6zc\" (UniqueName: \"kubernetes.io/projected/f911182b-3807-473b-b24a-59ee7610dd11-kube-api-access-kq6zc\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.115357 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.115267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911182b-3807-473b-b24a-59ee7610dd11-openshift-service-ca\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.115357 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.115292    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-metrics-certs\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.115984 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.115959    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911182b-3807-473b-b24a-59ee7610dd11-openshift-service-ca\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.117564 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.117544    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-maas-api-tls\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.117658 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.117638    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-metrics-certs\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.122003 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.121985    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq6zc\" (UniqueName: \"kubernetes.io/projected/f911182b-3807-473b-b24a-59ee7610dd11-kube-api-access-kq6zc\") pod \"maas-api-6545cd6846-2cldd\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") " pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.317476 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.317382    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:02.587365 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.587283    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"]
Sep 11 00:08:02.590791 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:08:02.590764    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf911182b_3807_473b_b24a_59ee7610dd11.slice/crio-f78d8c876950178ca70be9b9f226797e61374dae34ac94133c4e5abe6f226ccd WatchSource:0}: Error finding container f78d8c876950178ca70be9b9f226797e61374dae34ac94133c4e5abe6f226ccd: Status 404 returned error can't find the container with id f78d8c876950178ca70be9b9f226797e61374dae34ac94133c4e5abe6f226ccd
Sep 11 00:08:02.592199 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.592181    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 00:08:02.979622 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:02.979578    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95c53c5e-bb4d-4e02-a780-3262fc2e6da8" path="/var/lib/kubelet/pods/95c53c5e-bb4d-4e02-a780-3262fc2e6da8/volumes"
Sep 11 00:08:03.340468 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:03.340387    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd" event={"ID":"f911182b-3807-473b-b24a-59ee7610dd11","Type":"ContainerStarted","Data":"f78d8c876950178ca70be9b9f226797e61374dae34ac94133c4e5abe6f226ccd"}
Sep 11 00:08:05.353278 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:05.353245    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd" event={"ID":"f911182b-3807-473b-b24a-59ee7610dd11","Type":"ContainerStarted","Data":"28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145"}
Sep 11 00:08:05.353721 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:05.353358    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:05.370256 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:05.370207    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd" podStartSLOduration=1.691701608 podStartE2EDuration="4.370191212s" totalImagesPullingTime="2.678489604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:08:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:08:05.368798487 +0000 UTC m=+2099.022518656" watchObservedRunningTime="2026-09-11 00:08:05.370191212 +0000 UTC m=+2099.023911389"
Sep 11 00:08:07.164138 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:07.164103    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 11 00:08:07.165522 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:07.165500    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 11 00:08:07.170039 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:07.170021    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 11 00:08:07.171338 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:07.171318    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 11 00:08:07.514334 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:07.514247    2569 scope.go:117] "RemoveContainer" containerID="140fef472e7c526fe9996b6918c64fcd3e44293d184ced6cfbdcc6bf6bd67303"
Sep 11 00:08:10.092195 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.092156    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"]
Sep 11 00:08:10.092635 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.092380    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd" podUID="f911182b-3807-473b-b24a-59ee7610dd11" containerName="maas-api" containerID="cri-o://28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145" gracePeriod=30
Sep 11 00:08:10.099499 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.099477    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:10.153069 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.153036    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"]
Sep 11 00:08:10.157089 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.157062    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.166881 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.166851    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"]
Sep 11 00:08:10.199596 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.199542    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh9w8\" (UniqueName: \"kubernetes.io/projected/f02b47a3-feab-4451-b705-935ae7080712-kube-api-access-sh9w8\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.199781 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.199603    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f02b47a3-feab-4451-b705-935ae7080712-maas-api-tls\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.199781 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.199652    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f02b47a3-feab-4451-b705-935ae7080712-metrics-certs\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.199781 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.199720    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f02b47a3-feab-4451-b705-935ae7080712-openshift-service-ca\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.300629 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.300589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh9w8\" (UniqueName: \"kubernetes.io/projected/f02b47a3-feab-4451-b705-935ae7080712-kube-api-access-sh9w8\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.300824 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.300653    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f02b47a3-feab-4451-b705-935ae7080712-maas-api-tls\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.300824 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.300697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f02b47a3-feab-4451-b705-935ae7080712-metrics-certs\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.300824 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.300760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f02b47a3-feab-4451-b705-935ae7080712-openshift-service-ca\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.301686 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.301634    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f02b47a3-feab-4451-b705-935ae7080712-openshift-service-ca\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.304403 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.304366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f02b47a3-feab-4451-b705-935ae7080712-metrics-certs\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.305040 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.305018    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f02b47a3-feab-4451-b705-935ae7080712-maas-api-tls\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.309618 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.309595    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh9w8\" (UniqueName: \"kubernetes.io/projected/f02b47a3-feab-4451-b705-935ae7080712-kube-api-access-sh9w8\") pod \"maas-api-5db99c876b-p8jvg\" (UID: \"f02b47a3-feab-4451-b705-935ae7080712\") " pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.352959 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.352882    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:10.382729 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.382682    2569 generic.go:356] "Generic (PLEG): container finished" podID="f911182b-3807-473b-b24a-59ee7610dd11" containerID="28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145" exitCode=0
Sep 11 00:08:10.382729 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.382727    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"
Sep 11 00:08:10.383017 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.382772    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd" event={"ID":"f911182b-3807-473b-b24a-59ee7610dd11","Type":"ContainerDied","Data":"28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145"}
Sep 11 00:08:10.383017 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.382815    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6545cd6846-2cldd" event={"ID":"f911182b-3807-473b-b24a-59ee7610dd11","Type":"ContainerDied","Data":"f78d8c876950178ca70be9b9f226797e61374dae34ac94133c4e5abe6f226ccd"}
Sep 11 00:08:10.383017 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.382844    2569 scope.go:117] "RemoveContainer" containerID="28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145"
Sep 11 00:08:10.394074 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.394052    2569 scope.go:117] "RemoveContainer" containerID="28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145"
Sep 11 00:08:10.394358 ip-10-0-143-154 kubenswrapper[2569]: E0911 00:08:10.394341    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145\": container with ID starting with 28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145 not found: ID does not exist" containerID="28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145"
Sep 11 00:08:10.394426 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.394368    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145"} err="failed to get container status \"28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145\": rpc error: code = NotFound desc = could not find container \"28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145\": container with ID starting with 28665618332f137e83541aaf04d44eebfd3f6ab67d79c172a0b3aa644a486145 not found: ID does not exist"
Sep 11 00:08:10.401685 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.401663    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq6zc\" (UniqueName: \"kubernetes.io/projected/f911182b-3807-473b-b24a-59ee7610dd11-kube-api-access-kq6zc\") pod \"f911182b-3807-473b-b24a-59ee7610dd11\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") "
Sep 11 00:08:10.401786 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.401721    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911182b-3807-473b-b24a-59ee7610dd11-openshift-service-ca\") pod \"f911182b-3807-473b-b24a-59ee7610dd11\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") "
Sep 11 00:08:10.401851 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.401790    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-maas-api-tls\") pod \"f911182b-3807-473b-b24a-59ee7610dd11\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") "
Sep 11 00:08:10.401851 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.401838    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-metrics-certs\") pod \"f911182b-3807-473b-b24a-59ee7610dd11\" (UID: \"f911182b-3807-473b-b24a-59ee7610dd11\") "
Sep 11 00:08:10.402322 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.402297    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f911182b-3807-473b-b24a-59ee7610dd11-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f911182b-3807-473b-b24a-59ee7610dd11" (UID: "f911182b-3807-473b-b24a-59ee7610dd11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 00:08:10.404172 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.404146    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f911182b-3807-473b-b24a-59ee7610dd11" (UID: "f911182b-3807-473b-b24a-59ee7610dd11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:08:10.404326 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.404310    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f911182b-3807-473b-b24a-59ee7610dd11" (UID: "f911182b-3807-473b-b24a-59ee7610dd11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 00:08:10.404326 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.404315    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f911182b-3807-473b-b24a-59ee7610dd11-kube-api-access-kq6zc" (OuterVolumeSpecName: "kube-api-access-kq6zc") pod "f911182b-3807-473b-b24a-59ee7610dd11" (UID: "f911182b-3807-473b-b24a-59ee7610dd11"). InnerVolumeSpecName "kube-api-access-kq6zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 00:08:10.482597 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.482561    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:10.503495 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.503462    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq6zc\" (UniqueName: \"kubernetes.io/projected/f911182b-3807-473b-b24a-59ee7610dd11-kube-api-access-kq6zc\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:10.503641 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.503499    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911182b-3807-473b-b24a-59ee7610dd11-openshift-service-ca\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:10.503641 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.503517    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-maas-api-tls\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:10.503641 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.503532    2569 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911182b-3807-473b-b24a-59ee7610dd11-metrics-certs\") on node \"ip-10-0-143-154.ec2.internal\" DevicePath \"\""
Sep 11 00:08:10.701992 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.701960    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"]
Sep 11 00:08:10.705238 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.705205    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6545cd6846-2cldd"]
Sep 11 00:08:10.754896 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.754863    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"]
Sep 11 00:08:10.758969 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:08:10.758914    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf02b47a3_feab_4451_b705_935ae7080712.slice/crio-c8c5d99fc75a05070743db14feb9404011b211ce689b13d82c11881efa0872df WatchSource:0}: Error finding container c8c5d99fc75a05070743db14feb9404011b211ce689b13d82c11881efa0872df: Status 404 returned error can't find the container with id c8c5d99fc75a05070743db14feb9404011b211ce689b13d82c11881efa0872df
Sep 11 00:08:10.978771 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:10.978676    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f911182b-3807-473b-b24a-59ee7610dd11" path="/var/lib/kubelet/pods/f911182b-3807-473b-b24a-59ee7610dd11/volumes"
Sep 11 00:08:11.392074 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:11.392037    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg" event={"ID":"f02b47a3-feab-4451-b705-935ae7080712","Type":"ContainerStarted","Data":"e7e81ca1d7ef2c9b54ba3a1bbbcc2aef15375ecf40634895b4cd1c73e73eb593"}
Sep 11 00:08:11.392549 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:11.392087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg" event={"ID":"f02b47a3-feab-4451-b705-935ae7080712","Type":"ContainerStarted","Data":"c8c5d99fc75a05070743db14feb9404011b211ce689b13d82c11881efa0872df"}
Sep 11 00:08:11.392549 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:11.392128    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:11.407397 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:11.407351    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg" podStartSLOduration=0.851120294 podStartE2EDuration="1.407332566s" totalImagesPullingTime="556.212272ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 00:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:08:11.406500144 +0000 UTC m=+2105.060220316" watchObservedRunningTime="2026-09-11 00:08:11.407332566 +0000 UTC m=+2105.061052781"
Sep 11 00:08:17.401834 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:17.401802    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5db99c876b-p8jvg"
Sep 11 00:08:31.542150 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:31.542121    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-668cdb6579-l44hr_e8dbce6c-773e-41b1-9c53-cd025ac3246c/authorino/0.log"
Sep 11 00:08:36.665430 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:36.665396    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tf8w2_1582b7b7-3f82-4845-9992-46007b14da6b/manager/0.log"
Sep 11 00:08:36.779156 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:36.779127    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pnll2_eaacc499-1307-4a91-9f42-c17101a52b79/manager/0.log"
Sep 11 00:08:37.241775 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:37.241748    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc8465fcf-2qkrv_53514fe5-a017-408c-8251-34237340f3a5/copy-manifests/0.log"
Sep 11 00:08:37.253192 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:37.253160    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc8465fcf-2qkrv_53514fe5-a017-408c-8251-34237340f3a5/manager/0.log"
Sep 11 00:08:37.925262 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:37.925237    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5db99c876b-p8jvg_f02b47a3-feab-4451-b705-935ae7080712/maas-api/0.log"
Sep 11 00:08:38.033735 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:38.033710    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818065-m6hmn_1c8e97a4-b36c-4fa5-b439-85c7b0876202/cleanup/0.log"
Sep 11 00:08:38.139627 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:38.139600    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818080-rlzxm_80484705-6646-4448-8fff-4bbb9649e77f/cleanup/0.log"
Sep 11 00:08:39.003753 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:39.003708    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-668cdb6579-l44hr_e8dbce6c-773e-41b1-9c53-cd025ac3246c/authorino/0.log"
Sep 11 00:08:40.000877 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:40.000845    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh_ed4c0472-2c41-4c23-baca-d10f8d8e8094/istio-proxy/0.log"
Sep 11 00:08:40.445155 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:40.445120    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n8b68_1ddd5db5-9ef6-4dea-a625-36023eb3e937/istio-proxy/1.log"
Sep 11 00:08:40.555125 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:40.555095    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-67mjh_a66761c6-89c6-41fd-a2f2-5846a082df0d/payload-pre-processing/0.log"
Sep 11 00:08:40.774352 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:40.774278    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d8787b64-rmldg_217cfaa5-caf3-4e47-837c-c8dc90c53f63/router/0.log"
Sep 11 00:08:41.113853 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:41.113820    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-kx4mv_71cf3014-6733-413a-9ddc-2e988c7338e7/main/0.log"
Sep 11 00:08:50.356794 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:50.356762    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lgb42_4c5e1758-ac02-4c25-8c56-7f229ac909b7/global-pull-secret-syncer/0.log"
Sep 11 00:08:50.465392 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:50.465350    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l297b_29d288de-88fe-491c-a6fd-4edcd4cd5541/konnectivity-agent/0.log"
Sep 11 00:08:50.536397 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:50.536374    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-154.ec2.internal_ed43c03ec92d5b25a955c89d154a23b6/haproxy/0.log"
Sep 11 00:08:55.073568 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:55.073483    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-668cdb6579-l44hr_e8dbce6c-773e-41b1-9c53-cd025ac3246c/authorino/0.log"
Sep 11 00:08:56.981274 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:56.981245    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-xpr5q_c677bd31-57e0-4d9c-a478-f5d4d1373137/cluster-monitoring-operator/0.log"
Sep 11 00:08:57.213174 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:57.213147    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-djsm9_072fedc4-764f-4c72-a3f7-5a5b50bb6f9a/node-exporter/0.log"
Sep 11 00:08:57.232191 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:57.232127    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-djsm9_072fedc4-764f-4c72-a3f7-5a5b50bb6f9a/kube-rbac-proxy/0.log"
Sep 11 00:08:57.250357 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:57.250335    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-djsm9_072fedc4-764f-4c72-a3f7-5a5b50bb6f9a/init-textfile/0.log"
Sep 11 00:08:59.435091 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:08:59.435011    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-tjtxx_1378e9cb-d095-4c5c-ad0a-afd6b7e6f229/networking-console-plugin/0.log"
Sep 11 00:09:00.038403 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.038372    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/1.log"
Sep 11 00:09:00.043660 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.043637    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-8g9xq_0e399181-306e-43df-a4e6-c35ecdd9881d/console-operator/2.log"
Sep 11 00:09:00.607535 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.607509    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-vbzh6_0fcbe7f7-c497-4112-9b12-a96a3be661b0/download-server/0.log"
Sep 11 00:09:00.641460 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.641431    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"]
Sep 11 00:09:00.641793 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.641780    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f911182b-3807-473b-b24a-59ee7610dd11" containerName="maas-api"
Sep 11 00:09:00.641836 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.641794    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f911182b-3807-473b-b24a-59ee7610dd11" containerName="maas-api"
Sep 11 00:09:00.641878 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.641870    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f911182b-3807-473b-b24a-59ee7610dd11" containerName="maas-api"
Sep 11 00:09:00.644905 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.644890    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.647091 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.647072    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mzltk\"/\"openshift-service-ca.crt\""
Sep 11 00:09:00.647091 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.647087    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mzltk\"/\"default-dockercfg-cvh2v\""
Sep 11 00:09:00.647250 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.647093    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mzltk\"/\"kube-root-ca.crt\""
Sep 11 00:09:00.654046 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.654020    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"]
Sep 11 00:09:00.761689 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.761655    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-sys\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.761897 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.761709    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-podres\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.761897 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.761731    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-proc\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.761897 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.761789    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-lib-modules\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.761897 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.761854    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmjrd\" (UniqueName: \"kubernetes.io/projected/2d417422-adfa-4c52-8b31-b5ce68885f2a-kube-api-access-xmjrd\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863037 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-sys\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863218 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863066    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-podres\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863218 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-proc\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863218 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863132    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-sys\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863218 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863135    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-lib-modules\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863391 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmjrd\" (UniqueName: \"kubernetes.io/projected/2d417422-adfa-4c52-8b31-b5ce68885f2a-kube-api-access-xmjrd\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863391 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-podres\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863391 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863241    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-lib-modules\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.863391 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.863227    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d417422-adfa-4c52-8b31-b5ce68885f2a-proc\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.870309 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.870288    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmjrd\" (UniqueName: \"kubernetes.io/projected/2d417422-adfa-4c52-8b31-b5ce68885f2a-kube-api-access-xmjrd\") pod \"perf-node-gather-daemonset-8tpnb\" (UID: \"2d417422-adfa-4c52-8b31-b5ce68885f2a\") " pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:00.954676 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:00.954644    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:01.225185 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:01.225161    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"]
Sep 11 00:09:01.228304 ip-10-0-143-154 kubenswrapper[2569]: W0911 00:09:01.228281    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2d417422_adfa_4c52_8b31_b5ce68885f2a.slice/crio-56aff1a003596c3cc2d788966debbcb52c9a0416236a317de2cfb6265b41dd9b WatchSource:0}: Error finding container 56aff1a003596c3cc2d788966debbcb52c9a0416236a317de2cfb6265b41dd9b: Status 404 returned error can't find the container with id 56aff1a003596c3cc2d788966debbcb52c9a0416236a317de2cfb6265b41dd9b
Sep 11 00:09:01.342867 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:01.342832    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-65dwf_1d604ae4-a70a-4080-b57b-e1b80e804c4b/volume-data-source-validator/0.log"
Sep 11 00:09:01.633574 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:01.633541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb" event={"ID":"2d417422-adfa-4c52-8b31-b5ce68885f2a","Type":"ContainerStarted","Data":"4fe0741b22c130aa13ab45990e1bb4dbcb5bfe832a72a29573ece07104c7ab19"}
Sep 11 00:09:01.633996 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:01.633600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb" event={"ID":"2d417422-adfa-4c52-8b31-b5ce68885f2a","Type":"ContainerStarted","Data":"56aff1a003596c3cc2d788966debbcb52c9a0416236a317de2cfb6265b41dd9b"}
Sep 11 00:09:01.633996 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:01.633630    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:01.647229 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:01.647174    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb" podStartSLOduration=1.647154028 podStartE2EDuration="1.647154028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 00:09:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 00:09:01.646196191 +0000 UTC m=+2155.299916366" watchObservedRunningTime="2026-09-11 00:09:01.647154028 +0000 UTC m=+2155.300874202"
Sep 11 00:09:02.401602 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:02.401562    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5hczz_8aab137b-75bc-4e7d-b33a-590054eaa2f7/dns/0.log"
Sep 11 00:09:02.421018 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:02.420990    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5hczz_8aab137b-75bc-4e7d-b33a-590054eaa2f7/kube-rbac-proxy/0.log"
Sep 11 00:09:02.526876 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:02.526846    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4fj9t_f24f3d3d-137d-4393-b095-e651175daf4c/dns-node-resolver/0.log"
Sep 11 00:09:03.155407 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:03.155383    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vxkh_1e74baf0-dfa1-45b8-87ba-8fb556a20495/node-ca/0.log"
Sep 11 00:09:04.106040 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:04.106008    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh_ed4c0472-2c41-4c23-baca-d10f8d8e8094/istio-proxy/0.log"
Sep 11 00:09:04.387670 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:04.387631    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n8b68_1ddd5db5-9ef6-4dea-a625-36023eb3e937/istio-proxy/0.log"
Sep 11 00:09:04.395890 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:04.395870    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n8b68_1ddd5db5-9ef6-4dea-a625-36023eb3e937/istio-proxy/1.log"
Sep 11 00:09:04.426748 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:04.426721    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-67mjh_a66761c6-89c6-41fd-a2f2-5846a082df0d/payload-pre-processing/0.log"
Sep 11 00:09:04.483097 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:04.483057    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d8787b64-rmldg_217cfaa5-caf3-4e47-837c-c8dc90c53f63/router/0.log"
Sep 11 00:09:05.330432 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:05.330393    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g45l9_178c4a3f-841b-4d6c-8817-78ec6dc25085/serve-healthcheck-canary/0.log"
Sep 11 00:09:05.786594 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:05.786563    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-vm4gs_a993d388-bd24-4154-b232-2665a25ecab8/insights-operator/0.log"
Sep 11 00:09:05.787106 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:05.786963    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-vm4gs_a993d388-bd24-4154-b232-2665a25ecab8/insights-operator/1.log"
Sep 11 00:09:05.944773 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:05.944747    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ghqvk_cc42875d-dbfc-46c1-b4db-925d5ffe4c98/kube-rbac-proxy/0.log"
Sep 11 00:09:05.963783 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:05.963758    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ghqvk_cc42875d-dbfc-46c1-b4db-925d5ffe4c98/exporter/0.log"
Sep 11 00:09:05.982808 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:05.982778    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ghqvk_cc42875d-dbfc-46c1-b4db-925d5ffe4c98/extractor/0.log"
Sep 11 00:09:07.545148 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:07.545120    2569 scope.go:117] "RemoveContainer" containerID="fa03c9f76d1af0e59fe89df356ae63455292fd821fc02fe34ab9ec239c4ebb9c"
Sep 11 00:09:07.553365 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:07.553346    2569 scope.go:117] "RemoveContainer" containerID="31cda5d96b365a2c6260abeb929251f818786bf389d3ed3958df4fb186b5e25d"
Sep 11 00:09:07.652109 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:07.652081    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mzltk/perf-node-gather-daemonset-8tpnb"
Sep 11 00:09:07.993547 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:07.993517    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-tf8w2_1582b7b7-3f82-4845-9992-46007b14da6b/manager/0.log"
Sep 11 00:09:08.162180 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:08.162143    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pnll2_eaacc499-1307-4a91-9f42-c17101a52b79/manager/0.log"
Sep 11 00:09:08.396343 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:08.396310    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc8465fcf-2qkrv_53514fe5-a017-408c-8251-34237340f3a5/manager/0.log"
Sep 11 00:09:08.411184 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:08.411156    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc8465fcf-2qkrv_53514fe5-a017-408c-8251-34237340f3a5/copy-manifests/0.log"
Sep 11 00:09:09.809292 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:09.809211    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bffb6dd48-7zgnf_dbed31aa-6127-4291-8136-89624f6996d8/manager/0.log"
Sep 11 00:09:10.823709 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:10.823677    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh_ed4c0472-2c41-4c23-baca-d10f8d8e8094/istio-proxy/0.log"
Sep 11 00:09:11.088564 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:11.088472    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n8b68_1ddd5db5-9ef6-4dea-a625-36023eb3e937/istio-proxy/0.log"
Sep 11 00:09:11.096817 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:11.096789    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n8b68_1ddd5db5-9ef6-4dea-a625-36023eb3e937/istio-proxy/1.log"
Sep 11 00:09:11.118672 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:11.118641    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-67mjh_a66761c6-89c6-41fd-a2f2-5846a082df0d/payload-pre-processing/0.log"
Sep 11 00:09:11.165947 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:11.165918    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d8787b64-rmldg_217cfaa5-caf3-4e47-837c-c8dc90c53f63/router/0.log"
Sep 11 00:09:12.665929 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:12.665900    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-kctfh_ed4c0472-2c41-4c23-baca-d10f8d8e8094/istio-proxy/0.log"
Sep 11 00:09:12.930626 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:12.930543    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n8b68_1ddd5db5-9ef6-4dea-a625-36023eb3e937/istio-proxy/0.log"
Sep 11 00:09:12.939663 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:12.939623    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-n8b68_1ddd5db5-9ef6-4dea-a625-36023eb3e937/istio-proxy/1.log"
Sep 11 00:09:12.969348 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:12.969322    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-67mjh_a66761c6-89c6-41fd-a2f2-5846a082df0d/payload-pre-processing/0.log"
Sep 11 00:09:13.011633 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:13.011605    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64d8787b64-rmldg_217cfaa5-caf3-4e47-837c-c8dc90c53f63/router/0.log"
Sep 11 00:09:14.981679 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:14.981633    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-m2hgm_6d3dac95-bd90-44bf-bb9b-2837fcadcf58/migrator/0.log"
Sep 11 00:09:14.998955 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:14.998902    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-m2hgm_6d3dac95-bd90-44bf-bb9b-2837fcadcf58/graceful-termination/0.log"
Sep 11 00:09:15.334694 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:15.334593    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-cfdcc_abd2e556-839b-4d11-8c58-566ff67e0fd9/kube-storage-version-migrator-operator/1.log"
Sep 11 00:09:15.335433 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:15.335414    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-cfdcc_abd2e556-839b-4d11-8c58-566ff67e0fd9/kube-storage-version-migrator-operator/0.log"
Sep 11 00:09:16.362072 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.362043    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9qt4w_85a41d6c-95e7-4a39-99e6-6b5ff81acfc3/kube-multus/0.log"
Sep 11 00:09:16.660072 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.660046    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t6th9_256ec37b-d6e3-436b-8fba-c5046b103926/kube-multus-additional-cni-plugins/0.log"
Sep 11 00:09:16.682063 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.682037    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t6th9_256ec37b-d6e3-436b-8fba-c5046b103926/egress-router-binary-copy/0.log"
Sep 11 00:09:16.701256 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.701232    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t6th9_256ec37b-d6e3-436b-8fba-c5046b103926/cni-plugins/0.log"
Sep 11 00:09:16.720324 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.720299    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t6th9_256ec37b-d6e3-436b-8fba-c5046b103926/bond-cni-plugin/0.log"
Sep 11 00:09:16.738736 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.738713    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t6th9_256ec37b-d6e3-436b-8fba-c5046b103926/routeoverride-cni/0.log"
Sep 11 00:09:16.757668 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.757643    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t6th9_256ec37b-d6e3-436b-8fba-c5046b103926/whereabouts-cni-bincopy/0.log"
Sep 11 00:09:16.776021 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.775995    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-t6th9_256ec37b-d6e3-436b-8fba-c5046b103926/whereabouts-cni/0.log"
Sep 11 00:09:16.953353 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.953269    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mh7bp_5dd6ce58-5b02-4e04-9463-38ccc4773b66/network-metrics-daemon/0.log"
Sep 11 00:09:16.968894 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:16.968870    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mh7bp_5dd6ce58-5b02-4e04-9463-38ccc4773b66/kube-rbac-proxy/0.log"
Sep 11 00:09:18.132994 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.132961    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-controller/0.log"
Sep 11 00:09:18.147646 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.147618    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/1.log"
Sep 11 00:09:18.147841 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.147696    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovn-acl-logging/0.log"
Sep 11 00:09:18.163993 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.163968    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/kube-rbac-proxy-node/0.log"
Sep 11 00:09:18.184797 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.184773    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/kube-rbac-proxy-ovn-metrics/0.log"
Sep 11 00:09:18.204205 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.204183    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/northd/0.log"
Sep 11 00:09:18.223361 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.223330    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/nbdb/0.log"
Sep 11 00:09:18.242562 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.242542    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/sbdb/0.log"
Sep 11 00:09:18.297923 ip-10-0-143-154 kubenswrapper[2569]: I0911 00:09:18.297891    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zxbz2_2b0cbfcf-07ff-4607-939b-7eff0b21acf9/ovnkube-controller/0.log"