Sep 17 23:04:14.103727 ip-10-0-128-233 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 23:04:14.103740 ip-10-0-128-233 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 23:04:14.103750 ip-10-0-128-233 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 23:04:14.104065 ip-10-0-128-233 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 23:04:23.429482 ip-10-0-128-233 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 17 23:04:23.429494 ip-10-0-128-233 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 30a4a2fcc6cf4330adaa22111fb303ca --
Sep 17 23:06:18.632392 ip-10-0-128-233 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 23:06:21.800582 ip-10-0-128-233 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:06:21.800582 ip-10-0-128-233 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 23:06:21.800582 ip-10-0-128-233 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:06:21.800582 ip-10-0-128-233 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 23:06:21.800582 ip-10-0-128-233 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 23:06:21.801249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.800652    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 23:06:21.804982 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.804958    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:21.804982 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.804978    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:21.804982 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.804982    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:21.804982 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.804985    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:21.804982 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.804989    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.804992    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.804998    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805002    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805005    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805008    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805011    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805014    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805017    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805020    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805022    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805025    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805028    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805031    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805033    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805036    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805038    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805041    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805044    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:21.805228 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805047    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805049    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805052    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805054    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805057    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805060    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805064    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805067    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805071    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805077    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805079    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805082    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805085    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805087    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805090    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805093    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805096    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805099    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805102    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805104    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:21.805685 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805107    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805109    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805112    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805115    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805119    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805136    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805139    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805141    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805144    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805147    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805150    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805152    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805156    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805159    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805161    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805164    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805167    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805170    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805173    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805175    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:21.806193 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805178    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805181    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805184    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805187    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805189    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805192    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805195    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805223    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805227    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805230    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805234    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805237    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805240    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805244    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805247    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805251    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805253    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805256    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805259    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805262    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:21.806714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805267    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805269    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805272    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805275    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805278    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805281    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805284    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805287    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805289    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805292    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805295    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805298    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805301    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805304    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805307    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.805310    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806007    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806013    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806016    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806020    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:21.807212 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806024    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806028    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806031    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806034    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806036    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806039    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806042    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806045    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806047    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806050    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806052    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806055    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806058    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806061    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806064    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806066    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806069    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806071    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806074    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806076    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:21.807688 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806079    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806082    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806085    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806088    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806090    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806093    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806096    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806098    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806101    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806104    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806106    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806110    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806112    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806115    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806118    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806133    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806136    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806141    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806144    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:21.808232 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806147    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806150    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806153    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806155    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806158    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806161    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806163    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806167    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806170    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806173    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806176    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806179    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806181    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806184    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806187    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806190    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806193    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806196    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806198    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806201    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:21.808700 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806204    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806206    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806209    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806213    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806216    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806220    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806222    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806225    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806228    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806231    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806233    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806236    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806238    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806241    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806243    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806246    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806249    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806251    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806254    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:21.809211 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806257    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806261    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806264    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806268    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806270    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806274    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806276    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806279    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806282    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806284    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806287    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806290    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806292    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806295    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806297    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806300    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806302    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806678    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806684    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806687    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:21.809678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806690    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806694    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806697    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806700    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806702    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806705    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806707    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806710    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806713    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806715    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806718    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806721    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806724    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806728    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806731    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806733    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806736    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806739    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806741    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:21.810187 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806744    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806747    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806749    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806752    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806754    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806757    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806760    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806762    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806764    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806767    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806770    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806772    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806775    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806777    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806780    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806783    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806785    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806788    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806790    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806793    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:21.810650 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806795    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806798    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806800    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806803    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806806    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806809    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806812    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806817    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806820    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806823    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806826    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806829    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806831    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806834    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806837    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806839    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806842    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806844    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806847    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806849    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:21.811148 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806852    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806854    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806857    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806860    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806864    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806867    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806869    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806872    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806876    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806879    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806882    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806884    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806887    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806889    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806892    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806894    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806897    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806899    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806902    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:21.811675 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806905    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806908    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806911    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806913    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806916    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806918    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806921    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806923    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806926    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806928    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806932    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806935    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806938    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806941    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806944    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806947    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806950    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.806952    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807024    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807058    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 23:06:21.812139 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807066    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807075    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807079    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807085    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807090    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807094    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807098    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807101    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807105    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807110    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807113    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807116    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807119    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807135    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807139    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807142    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807147    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807149    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807153    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807156    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807159    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807163    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807166    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 23:06:21.812632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807169    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807173    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807176    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807179    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807182    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807185    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807188    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807192    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807196    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807199    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807202    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807205    2581 flags.go:64] FLAG: --enable-server="true"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807210    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807214    2581 flags.go:64] FLAG: --event-burst="100"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807218    2581 flags.go:64] FLAG: --event-qps="50"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807221    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807224    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807227    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807234    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807237    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807240    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807243    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807246    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807249    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807253    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 23:06:21.813227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807256    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807259    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807262    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807265    2581 flags.go:64] FLAG: --feature-gates=""
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807269    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807272    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807275    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807278    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807281    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807284    2581 flags.go:64] FLAG: --help="false"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807287    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-128-233.ec2.internal"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807290    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807293    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807297    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807300    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807303    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807306    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807309    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807312    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807316    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807319    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807322    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807325    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807328    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 23:06:21.813844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807331    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807335    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807338    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807341    2581 flags.go:64] FLAG: --lock-file=""
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807344    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807347    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807350    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807355    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807364    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807367    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807370    2581 flags.go:64] FLAG: --logging-format="text"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807373    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807377    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807379    2581 flags.go:64] FLAG: --manifest-url=""
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807382    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807391    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807394    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807398    2581 flags.go:64] FLAG: --max-pods="110"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807401    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807404    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807407    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807409    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807412    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807415    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807418    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 23:06:21.814562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807425    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807429    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807432    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807435    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807438    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807443    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807446    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807449    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807452    2581 flags.go:64] FLAG: --port="10250"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807457    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807460    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03bb10ede361ef90c"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807463    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807466    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807469    2581 flags.go:64] FLAG: --register-node="true"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807472    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807475    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807479    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807482    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807484    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807488    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807491    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807494    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807497    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807500    2581 flags.go:64] FLAG: --runonce="false"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807503    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 23:06:21.815252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807506    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807508    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807512    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807514    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807517    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807520    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807524    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807527    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807529    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807533    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807535    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807539    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807541    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807544    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807549    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807552    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807557    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807561    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807564    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807567    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807570    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807573    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807576    2581 flags.go:64] FLAG: --v="2"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807580    2581 flags.go:64] FLAG: --version="false"
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807584    2581 flags.go:64] FLAG: --vmodule=""
Sep 17 23:06:21.815930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807589    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.807592    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807710    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807714    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807718    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807721    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807723    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807726    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807729    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807731    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807736    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807740    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807742    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807745    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807748    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807751    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807754    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807758    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807761    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:21.816631 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807764    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807767    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807770    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807773    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807777    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807780    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807783    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807785    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807788    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807791    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807793    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807796    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807798    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807801    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807804    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807807    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807809    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807812    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807814    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807817    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:21.817209 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807820    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807822    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807825    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807827    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807830    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807833    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807835    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807838    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807840    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807844    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807847    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807849    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807852    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807854    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807857    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807859    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807863    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807866    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807868    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807872    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:21.817769 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807876    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807879    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807881    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807884    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807886    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807889    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807892    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807895    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807897    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807900    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807903    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807906    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807908    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807911    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807914    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807916    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807919    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807922    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807924    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:21.818337 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807927    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807929    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807933    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807936    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807939    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807941    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807944    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807947    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807949    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807953    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807955    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807958    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807961    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807963    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807966    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807968    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807971    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807974    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807976    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807979    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:21.818816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807985    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807988    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.807990    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.808720    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.815498    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.815515    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.815534    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.815544    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815598    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815602    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815606    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815609    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815612    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815615    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815617    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:21.819329 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815620    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815623    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815627    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815629    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815632    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815634    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815637    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815640    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815643    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815646    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815648    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815651    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815654    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815657    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815660    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815662    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815665    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815668    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815670    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815673    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:21.819718 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815676    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815678    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815683    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815686    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815689    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815692    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815695    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815697    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815700    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815703    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815706    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815708    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815711    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815714    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815716    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815719    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815722    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815724    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815727    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815730    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:21.820220 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815733    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815735    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815738    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815741    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815743    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815746    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815749    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815752    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815755    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815759    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815763    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815766    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815769    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815772    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815775    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815779    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815781    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815784    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815787    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815789    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:21.820747 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815792    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815795    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815798    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815801    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815803    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815806    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815809    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815812    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815815    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815817    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815822    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815825    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815829    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815833    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815837    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815840    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815843    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815846    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815849    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:21.821255 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815852    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815856    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815859    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815862    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815865    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815867    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815870    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815874    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815877    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815880    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815883    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815885    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.815888    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.815893    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816003    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 23:06:21.821716 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816007    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816010    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816013    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816016    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816018    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816021    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816024    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816027    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816029    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816032    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816035    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816038    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816041    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816043    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816046    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816049    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816052    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816055    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816057    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816060    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 23:06:21.822094 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816062    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816065    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816068    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816070    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816074    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816076    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816079    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816082    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816085    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816088    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816090    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816093    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816095    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816099    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816102    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816105    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816108    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816110    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816113    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 23:06:21.822825 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816115    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816118    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816135    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816139    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816142    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816144    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816147    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816150    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816152    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816155    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816158    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816161    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816165    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816168    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816171    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816174    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816177    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816180    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816184    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816186    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 23:06:21.823511 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816189    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816196    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816199    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816202    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816204    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816207    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816209    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816212    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816215    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816217    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816220    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816222    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816225    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816227    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816230    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816232    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816235    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816239    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816242    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816245    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 23:06:21.824629 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816248    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816251    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816254    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816257    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816260    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816262    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816265    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816268    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816271    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816273    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816276    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816279    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816281    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816284    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816287    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816290    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816293    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816295    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 23:06:21.825116 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:21.816298    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 23:06:21.825570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.816302    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 23:06:21.825570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.816405    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 23:06:21.825570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.818403    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 23:06:21.825570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.819154    2581 server.go:1019] "Starting client certificate rotation"
Sep 17 23:06:21.825570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.819252    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 23:06:21.825570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.819295    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 23:06:21.825570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.821193    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 23:06:21.825570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.824856    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 23:06:21.828484 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.828468    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 17 23:06:21.830003 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.829988    2581 log.go:25] "Validated CRI v1 image API"
Sep 17 23:06:21.830775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.830759    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 23:06:21.833532 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.833515    2581 fs.go:135] Filesystem UUIDs: map[5baadc16-917d-4f76-8d52-518c599cbead:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b31d6dcc-e66b-4be2-bb05-508208cf9b91:/dev/nvme0n1p4]
Sep 17 23:06:21.833581 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.833533    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 23:06:21.837884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.837765    2581 manager.go:216] Machine: {Timestamp:2026-09-17 23:06:21.837300356 +0000 UTC m=+1.841492185 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100269 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2db7b65365934eb15bb9173165911e SystemUUID:ec2db7b6-5365-934e-b15b-b9173165911e BootID:30a4a2fc-c6cf-4330-adaa-22111fb303ca Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e9:fb:5a:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e9:fb:5a:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:b2:ec:45:4d:40 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 23:06:21.837884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.837876    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 23:06:21.838006 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.837953    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 23:06:21.839653 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.839628    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 23:06:21.839787 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.839657    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-233.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 23:06:21.840795 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.840786    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 23:06:21.840832 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.840798    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 23:06:21.840832 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.840806    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 23:06:21.840832 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.840817    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 23:06:21.840920 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.840884    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 23:06:21.844558 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.844546    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 23:06:21.844662 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.844653    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 23:06:21.844695 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.844688    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 23:06:21.844731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.844699    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 23:06:21.844731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.844713    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 23:06:21.844731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.844722    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 17 23:06:21.844840 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.844731    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 23:06:21.845102 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.845089    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 23:06:21.845150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.845113    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 23:06:21.846883 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.846862    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 23:06:21.849042 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.849028    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 23:06:21.849104 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.849058    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 23:06:21.850310 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850297    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 23:06:21.850350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850314    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 23:06:21.850350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850321    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 23:06:21.850350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850327    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 23:06:21.850350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850336    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 23:06:21.850350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850342    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 23:06:21.850503 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850355    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 23:06:21.850503 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850360    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 23:06:21.850503 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850367    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 23:06:21.850503 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850372    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 23:06:21.850503 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850386    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 23:06:21.850503 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850391    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 23:06:21.850503 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850415    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 23:06:21.850503 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.850420    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 23:06:21.854087 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.854069    2581 server.go:1268] "Started kubelet"
Sep 17 23:06:21.854704 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.854438    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 23:06:21.854811 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.854758    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 23:06:21.854861 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.854849    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 23:06:21.856203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.856182    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 23:06:21.858452 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.856452    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 23:06:21.858452 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.856613    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 23:06:21.856561 ip-10-0-128-233 systemd[1]: Started Kubernetes Kubelet.
Sep 17 23:06:21.859510 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.859356    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 23:06:21.859686 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.859664    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 23:06:21.859742 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.859722    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 23:06:21.859813 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.859799    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-233.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 23:06:21.862632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.862615    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 23:06:21.862707 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.862628    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 23:06:21.862772 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.862755    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:21.863176 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.863157    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 23:06:21.863176 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.863177    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 23:06:21.863317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.863216    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 23:06:21.863317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.863286    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 23:06:21.863317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.863292    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 23:06:21.872077 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.872051    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-233.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 23:06:21.872168 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.872106    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 23:06:21.876495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.876479    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8k7s7"
Sep 17 23:06:21.883772 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.883754    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8k7s7"
Sep 17 23:06:21.885394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.885368    2581 factory.go:55] Registering systemd factory
Sep 17 23:06:21.885394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.885390    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 17 23:06:21.885667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.885650    2581 factory.go:153] Registering CRI-O factory
Sep 17 23:06:21.885667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.885667    2581 factory.go:223] Registration of the crio container factory successfully
Sep 17 23:06:21.885820 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.885715    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 23:06:21.885820 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.885729    2581 factory.go:103] Registering Raw factory
Sep 17 23:06:21.885820 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.885745    2581 manager.go:1196] Started watching for new ooms in manager
Sep 17 23:06:21.886243 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.886231    2581 manager.go:349] Starting recovery of all containers
Sep 17 23:06:21.890364 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.889467    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-233.ec2.internal.18d63e4839c3ad80  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-233.ec2.internal,UID:ip-10-0-128-233.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-233.ec2.internal,},FirstTimestamp:2026-09-17 23:06:21.85404352 +0000 UTC m=+1.858235351,LastTimestamp:2026-09-17 23:06:21.85404352 +0000 UTC m=+1.858235351,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-233.ec2.internal,}"
Sep 17 23:06:21.896086 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.896069    2581 manager.go:354] Recovery completed
Sep 17 23:06:21.899902 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.899888    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:21.901761 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.901748    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:21.901818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.901779    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:21.901818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.901792    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:21.902350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.902338    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 23:06:21.902350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.902350    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 23:06:21.902444 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.902361    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:21.902444 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.902364    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 23:06:21.903053 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.903039    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:21.903111 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.903064    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:21.903111 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.903077    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:21.904490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.904479    2581 policy_none.go:49] "None policy: Start"
Sep 17 23:06:21.904523 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.904494    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 23:06:21.904523 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.904505    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 23:06:21.906514 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.906503    2581 policy_none.go:47] "Start"
Sep 17 23:06:21.952950 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.952769    2581 manager.go:337] "Starting Device Plugin manager"
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.952988    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.953005    2581 server.go:87] "Starting device plugin registration server"
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.953296    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.953317    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.953411    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.953497    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.953506    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.954236    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 23:06:21.964947 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.954290    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:21.985596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.985570    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 23:06:21.986795 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.986772    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 23:06:21.986889 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.986818    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 23:06:21.986889 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.986843    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 23:06:21.986985 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:21.986892    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 23:06:21.988831 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:21.988813    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:22.053610 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.053539    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:22.054429 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.054414    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:22.054519 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.054448    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:22.054519 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.054464    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:22.054519 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.054492    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.059928 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.059914    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.060012 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.059939    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-233.ec2.internal\": node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:22.079495 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.079467    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:22.088276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.087359    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal"]
Sep 17 23:06:22.088359 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.088314    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:22.089099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.089077    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:22.089194 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.089110    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:22.089194 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.089138    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:22.091376 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.091363    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:22.091509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.091492    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.091551 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.091529    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:22.092071 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.092053    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:22.092171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.092072    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:22.092171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.092083    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:22.092171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.092095    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:22.092171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.092099    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:22.092171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.092114    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:22.094346 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.094329    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.094420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.094355    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 23:06:22.096276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.096258    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 23:06:22.096341 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.096290    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 23:06:22.096341 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.096302    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeHasSufficientPID"
Sep 17 23:06:22.125734 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.125715    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-233.ec2.internal\" not found" node="ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.130262 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.130247    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-233.ec2.internal\" not found" node="ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.180115 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.180094    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:22.264668 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.264643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5760489518c11caa7e8d239fc8d15015-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal\" (UID: \"5760489518c11caa7e8d239fc8d15015\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.264668 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.264671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5760489518c11caa7e8d239fc8d15015-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal\" (UID: \"5760489518c11caa7e8d239fc8d15015\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.264851 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.264696    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6989cb969ca76f3d903d482174d8453-config\") pod \"kube-apiserver-proxy-ip-10-0-128-233.ec2.internal\" (UID: \"e6989cb969ca76f3d903d482174d8453\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.280746 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.280722    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:22.365389 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.365360    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6989cb969ca76f3d903d482174d8453-config\") pod \"kube-apiserver-proxy-ip-10-0-128-233.ec2.internal\" (UID: \"e6989cb969ca76f3d903d482174d8453\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.365502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.365398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5760489518c11caa7e8d239fc8d15015-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal\" (UID: \"5760489518c11caa7e8d239fc8d15015\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.365502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.365419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5760489518c11caa7e8d239fc8d15015-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal\" (UID: \"5760489518c11caa7e8d239fc8d15015\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.365502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.365454    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6989cb969ca76f3d903d482174d8453-config\") pod \"kube-apiserver-proxy-ip-10-0-128-233.ec2.internal\" (UID: \"e6989cb969ca76f3d903d482174d8453\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.365502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.365463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5760489518c11caa7e8d239fc8d15015-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal\" (UID: \"5760489518c11caa7e8d239fc8d15015\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.365502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.365465    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5760489518c11caa7e8d239fc8d15015-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal\" (UID: \"5760489518c11caa7e8d239fc8d15015\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.381483 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.381459    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:22.429637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.429615    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.433783 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.433768    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.482585 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.482553    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:22.583077 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.583030    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:22.683624 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.683553    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-233.ec2.internal\" not found"
Sep 17 23:06:22.724577 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.724557    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:22.732932 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.732916    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:22.763696 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.763676    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.774579 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.774564    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 23:06:22.776103 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.776091    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal"
Sep 17 23:06:22.782570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.782557    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 23:06:22.819596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.819571    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 23:06:22.820027 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.819703    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 23:06:22.820027 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.819726    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 23:06:22.820027 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.819741    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 23:06:22.845886 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.845866    2581 apiserver.go:52] "Watching apiserver"
Sep 17 23:06:22.862512 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.862486    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 23:06:22.862768 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.862747    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 23:06:22.862901 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.862880    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-csnkx","kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal","openshift-dns/node-resolver-46t4x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal","openshift-multus/multus-ntq4l","openshift-multus/network-metrics-daemon-khgbr","openshift-network-diagnostics/network-check-target-xfshv","openshift-network-operator/iptables-alerter-tl28n","kube-system/konnectivity-agent-jq5sx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg","openshift-cluster-node-tuning-operator/tuned-xk96t","openshift-image-registry/node-ca-7287k","openshift-multus/multus-additional-cni-plugins-dbnmm","openshift-ovn-kubernetes/ovnkube-node-mllxt"]
Sep 17 23:06:22.865478 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.865461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.865561 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.865525    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:22.867583 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.867568    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:22.868477 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.868459    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-kubelet-config\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.868585 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.868481    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-dbus\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.868585 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.868496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.869787 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.869767    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 23:06:22.869890 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.869803    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.869951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.869892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hqrmh\""
Sep 17 23:06:22.870000 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.869958    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 23:06:22.872171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.872109    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 23:06:22.872363 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.872332    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 23:06:22.872481 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.872436    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 23:06:22.872685 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.872669    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 23:06:22.872744 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.872712    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wv5r5\""
Sep 17 23:06:22.873294 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.873269    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 23:06:22.873697 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.873682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:22.873748 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.873734    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:22.875769 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.875751    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:22.875852 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.875814    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:22.877881 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.877865    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:22.880020 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.880004    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:22.881471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.881304    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 23:06:22.881471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.881456    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 23:06:22.881587 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.881456    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pt6wz\""
Sep 17 23:06:22.881634 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.881596    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:06:22.882216 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.882201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:22.882599 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.882584    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 23:06:22.882651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.882606    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4kjgr\""
Sep 17 23:06:22.883330 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.883312    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 23:06:22.884425 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.884395    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.884587 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.884573    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqpvp\""
Sep 17 23:06:22.884658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.884616    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 23:06:22.884765 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.884753    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 23:06:22.885197 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.885184    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 23:06:22.886710 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.886692    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-52fs4\""
Sep 17 23:06:22.886766 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.886712    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 23:06:22.886824 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.886766    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:22.886824 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.886790    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:06:22.889109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.889092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:22.889109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.889106    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 23:06:22.889260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.889149    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cr9qc\""
Sep 17 23:06:22.889260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.889172    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 23:06:22.889260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.889200    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 23:06:22.889408 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.889260    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 23:01:21 +0000 UTC" deadline="2028-03-01 14:56:49.522787784 +0000 UTC"
Sep 17 23:06:22.889408 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.889297    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12735h50m26.633493949s"
Sep 17 23:06:22.891260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.891242    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dx72d\""
Sep 17 23:06:22.891472 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.891457    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 23:06:22.891556 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.891493    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.891617 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.891560    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 23:06:22.893771 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.893747    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 23:06:22.893771 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.893748    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-77wxj\""
Sep 17 23:06:22.893986 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.893971    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 23:06:22.894041 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.894027    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 23:06:22.894094 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.894056    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 23:06:22.894297 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.894281    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 23:06:22.895200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.895184    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 23:06:22.896049 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.896030    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-84v4r"
Sep 17 23:06:22.903185 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.903170    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-84v4r"
Sep 17 23:06:22.964153 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.964137    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 23:06:22.968729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968710    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-cni-bin\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.968775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-etc-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.968775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-lib-modules\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.968837 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968777    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-cni-binary-copy\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:22.968837 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968808    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwzpj\" (UniqueName: \"kubernetes.io/projected/e8b6666b-4910-4bd4-a3a3-f1a086a67052-kube-api-access-jwzpj\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.968837 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968829    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dxt9\" (UniqueName: \"kubernetes.io/projected/9f1c5761-f83c-4f1c-8994-8de155256444-kube-api-access-2dxt9\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:22.968931 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968843    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-var-lib-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.968931 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f06477c-095c-4f4b-9dec-1ba898a0c47b-iptables-alerter-script\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:22.968931 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:22.968931 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-modprobe-d\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.969044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-systemd\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.969044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968954    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:22.969044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:22.969044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.968991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsm5p\" (UniqueName: \"kubernetes.io/projected/1049d3e7-bedd-4ac4-b35d-f156a480d857-kube-api-access-hsm5p\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:22.969044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969004    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1695e8f9-ec8b-41db-a230-369fdae10dcb-tmp-dir\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:22.969044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-cni-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969054    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-kubernetes\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysctl-conf\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969087    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-host\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7cd72f17-5179-44c0-a06f-d330a2766e78-serviceca\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969142    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-netns\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969160    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-cni-multus\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-kubelet\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969203    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-slash\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.969276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969257    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-tmp\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969300    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-cni-netd\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysconfig\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-cnibin\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-system-cni-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-log-socket\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969474    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovnkube-config\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969533    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7cd72f17-5179-44c0-a06f-d330a2766e78-host\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-daemon-config\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-ovn\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f06477c-095c-4f4b-9dec-1ba898a0c47b-host-slash\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-tuned\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.969677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-cnibin\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-kubelet\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-cni-bin\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-env-overrides\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-socket-dir-parent\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969810    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-systemd-units\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969845    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-systemd\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovn-node-metrics-cert\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mnkx\" (UniqueName: \"kubernetes.io/projected/0f06477c-095c-4f4b-9dec-1ba898a0c47b-kube-api-access-7mnkx\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969926    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-socket-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv98s\" (UniqueName: \"kubernetes.io/projected/c47efd49-c2f8-4938-9a7b-7e32fc512951-kube-api-access-pv98s\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969974    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysctl-d\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.969991    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-var-lib-kubelet\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-hostroot\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.970230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970020    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-node-log\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89970b05-164f-41fc-b269-0ae42eeddb40-agent-certs\") pod \"konnectivity-agent-jq5sx\" (UID: \"89970b05-164f-41fc-b269-0ae42eeddb40\") " pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-sys-fs\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-etc-kubernetes\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970084    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9vrl\" (UniqueName: \"kubernetes.io/projected/070d5e2b-0645-4ed5-89e5-d500952f8dbd-kube-api-access-q9vrl\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970116    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89970b05-164f-41fc-b269-0ae42eeddb40-konnectivity-ca\") pod \"konnectivity-agent-jq5sx\" (UID: \"89970b05-164f-41fc-b269-0ae42eeddb40\") " pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970153    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-registration-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970197    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-system-cni-dir\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970216    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-k8s-cni-cncf-io\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970229    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bprd9\" (UniqueName: \"kubernetes.io/projected/7cd72f17-5179-44c0-a06f-d330a2766e78-kube-api-access-bprd9\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970244    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1695e8f9-ec8b-41db-a230-369fdae10dcb-hosts-file\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl5b6\" (UniqueName: \"kubernetes.io/projected/1695e8f9-ec8b-41db-a230-369fdae10dcb-kube-api-access-gl5b6\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8b6666b-4910-4bd4-a3a3-f1a086a67052-cni-binary-copy\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970309    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-sys\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970335    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovnkube-script-lib\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.970848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-dbus\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970428    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-run\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970451    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l26b\" (UniqueName: \"kubernetes.io/projected/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-kube-api-access-2l26b\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-conf-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-multus-certs\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.970518    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970521    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-run-netns\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970573    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-dbus\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:22.970610    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret podName:50b8ad4b-7ba8-4b82-9a42-6acdf46910b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:23.470584813 +0000 UTC m=+3.474776644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret") pod "global-pull-secret-syncer-csnkx" (UID: "50b8ad4b-7ba8-4b82-9a42-6acdf46910b8") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970625    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-kubelet-config\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-kubelet-config\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:22.971331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:22.970671    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-device-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.071601 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071565    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071605    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-modprobe-d\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071626    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-systemd\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071691    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-systemd\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071705    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsm5p\" (UniqueName: \"kubernetes.io/projected/1049d3e7-bedd-4ac4-b35d-f156a480d857-kube-api-access-hsm5p\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071679    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071732    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-modprobe-d\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071725    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1695e8f9-ec8b-41db-a230-369fdae10dcb-tmp-dir\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:23.071803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-cni-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-kubernetes\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysctl-conf\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-host\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071864    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-cni-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071919    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7cd72f17-5179-44c0-a06f-d330a2766e78-serviceca\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-host\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-netns\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-kubernetes\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-cni-multus\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.071995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-kubelet\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072022    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-slash\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072025    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1695e8f9-ec8b-41db-a230-369fdae10dcb-tmp-dir\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072028    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysctl-conf\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072043    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-netns\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072071    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-cni-multus\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-tmp\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.072317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072102    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072106    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-slash\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-cni-netd\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072076    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-kubelet\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysconfig\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-cni-netd\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-cnibin\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.072218    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072227    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysconfig\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-system-cni-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-cnibin\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.072274    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs podName:9f1c5761-f83c-4f1c-8994-8de155256444 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:23.572256821 +0000 UTC m=+3.576448655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs") pod "network-metrics-daemon-khgbr" (UID: "9f1c5761-f83c-4f1c-8994-8de155256444") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072291    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-system-cni-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-log-socket\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072326    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.073914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072360    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-log-socket\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072327    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072388    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovnkube-config\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072395    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072353    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7cd72f17-5179-44c0-a06f-d330a2766e78-host\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-daemon-config\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7cd72f17-5179-44c0-a06f-d330a2766e78-serviceca\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072507    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7cd72f17-5179-44c0-a06f-d330a2766e78-host\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-ovn\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f06477c-095c-4f4b-9dec-1ba898a0c47b-host-slash\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-ovn\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-tuned\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-cnibin\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0f06477c-095c-4f4b-9dec-1ba898a0c47b-host-slash\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-kubelet\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072650    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-cni-bin\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-env-overrides\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.074738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-cnibin\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-socket-dir-parent\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072748    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-cni-bin\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-socket-dir-parent\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-systemd-units\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-kubelet\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-systemd\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072845    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-systemd-units\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072856    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-etc-selinux\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-run-systemd\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072898    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovn-node-metrics-cert\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072925    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mnkx\" (UniqueName: \"kubernetes.io/projected/0f06477c-095c-4f4b-9dec-1ba898a0c47b-kube-api-access-7mnkx\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072927    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-run-ovn-kubernetes\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-socket-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.072977    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv98s\" (UniqueName: \"kubernetes.io/projected/c47efd49-c2f8-4938-9a7b-7e32fc512951-kube-api-access-pv98s\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.075622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.074399    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovnkube-config\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.074811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysctl-d\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.074872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-var-lib-kubelet\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.074900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-hostroot\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.074929    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-node-log\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.074961    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89970b05-164f-41fc-b269-0ae42eeddb40-agent-certs\") pod \"konnectivity-agent-jq5sx\" (UID: \"89970b05-164f-41fc-b269-0ae42eeddb40\") " pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.074996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-sys-fs\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-etc-kubernetes\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9vrl\" (UniqueName: \"kubernetes.io/projected/070d5e2b-0645-4ed5-89e5-d500952f8dbd-kube-api-access-q9vrl\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075310    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-socket-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-env-overrides\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075402    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-daemon-config\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075479    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89970b05-164f-41fc-b269-0ae42eeddb40-konnectivity-ca\") pod \"konnectivity-agent-jq5sx\" (UID: \"89970b05-164f-41fc-b269-0ae42eeddb40\") " pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-registration-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-system-cni-dir\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075615    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-k8s-cni-cncf-io\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.076280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bprd9\" (UniqueName: \"kubernetes.io/projected/7cd72f17-5179-44c0-a06f-d330a2766e78-kube-api-access-bprd9\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1695e8f9-ec8b-41db-a230-369fdae10dcb-hosts-file\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl5b6\" (UniqueName: \"kubernetes.io/projected/1695e8f9-ec8b-41db-a230-369fdae10dcb-kube-api-access-gl5b6\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8b6666b-4910-4bd4-a3a3-f1a086a67052-cni-binary-copy\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-sys\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovnkube-script-lib\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075827    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-run\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l26b\" (UniqueName: \"kubernetes.io/projected/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-kube-api-access-2l26b\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-conf-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-multus-certs\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.075973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-run-netns\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076003    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-device-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-cni-bin\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-etc-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-lib-modules\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-cni-binary-copy\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.076779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwzpj\" (UniqueName: \"kubernetes.io/projected/e8b6666b-4910-4bd4-a3a3-f1a086a67052-kube-api-access-jwzpj\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076199    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dxt9\" (UniqueName: \"kubernetes.io/projected/9f1c5761-f83c-4f1c-8994-8de155256444-kube-api-access-2dxt9\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-var-lib-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076263    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f06477c-095c-4f4b-9dec-1ba898a0c47b-iptables-alerter-script\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-tuned\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-tmp\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076683    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-multus-conf-dir\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.076755    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-multus-certs\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89970b05-164f-41fc-b269-0ae42eeddb40-konnectivity-ca\") pod \"konnectivity-agent-jq5sx\" (UID: \"89970b05-164f-41fc-b269-0ae42eeddb40\") " pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-host-run-netns\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077083    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-registration-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-device-dir\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077178    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-var-lib-cni-bin\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.077261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077222    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-hostroot\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.077614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c47efd49-c2f8-4938-9a7b-7e32fc512951-sys-fs\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.077614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.077672 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077649    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-var-lib-kubelet\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.077708 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1049d3e7-bedd-4ac4-b35d-f156a480d857-system-cni-dir\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.077839 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077812    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-host-run-k8s-cni-cncf-io\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.077936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.077920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-etc-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.078112 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.078089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1695e8f9-ec8b-41db-a230-369fdae10dcb-hosts-file\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:23.078280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.078261    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-lib-modules\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.078781 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.078747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8b6666b-4910-4bd4-a3a3-f1a086a67052-etc-kubernetes\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.078781 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.078774    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e8b6666b-4910-4bd4-a3a3-f1a086a67052-cni-binary-copy\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.078953 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.078852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-sys\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.078953 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.078929    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovn-node-metrics-cert\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.079097 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.079027    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-var-lib-openvswitch\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.079330 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.079314    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-run\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.079464 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.079426    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/070d5e2b-0645-4ed5-89e5-d500952f8dbd-ovnkube-script-lib\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.079544 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.079521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/070d5e2b-0645-4ed5-89e5-d500952f8dbd-node-log\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.079732 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.079711    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-etc-sysctl-d\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.080077 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.079821    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0f06477c-095c-4f4b-9dec-1ba898a0c47b-iptables-alerter-script\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:23.080315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.080293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1049d3e7-bedd-4ac4-b35d-f156a480d857-cni-binary-copy\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.081230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.081210    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89970b05-164f-41fc-b269-0ae42eeddb40-agent-certs\") pod \"konnectivity-agent-jq5sx\" (UID: \"89970b05-164f-41fc-b269-0ae42eeddb40\") " pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:23.082456 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.082435    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsm5p\" (UniqueName: \"kubernetes.io/projected/1049d3e7-bedd-4ac4-b35d-f156a480d857-kube-api-access-hsm5p\") pod \"multus-additional-cni-plugins-dbnmm\" (UID: \"1049d3e7-bedd-4ac4-b35d-f156a480d857\") " pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.085177 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.085009    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mnkx\" (UniqueName: \"kubernetes.io/projected/0f06477c-095c-4f4b-9dec-1ba898a0c47b-kube-api-access-7mnkx\") pod \"iptables-alerter-tl28n\" (UID: \"0f06477c-095c-4f4b-9dec-1ba898a0c47b\") " pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:23.085839 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.085644    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:23.085839 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.085669    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:23.085839 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.085683    2581 projected.go:196] Error preparing data for projected volume kube-api-access-k6xv2 for pod openshift-network-diagnostics/network-check-target-xfshv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:23.085839 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.085752    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2 podName:89932788-5773-4bce-817e-a0041862d6a9 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:23.585735371 +0000 UTC m=+3.589927205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k6xv2" (UniqueName: "kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2") pod "network-check-target-xfshv" (UID: "89932788-5773-4bce-817e-a0041862d6a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:23.087255 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.087228    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl5b6\" (UniqueName: \"kubernetes.io/projected/1695e8f9-ec8b-41db-a230-369fdae10dcb-kube-api-access-gl5b6\") pod \"node-resolver-46t4x\" (UID: \"1695e8f9-ec8b-41db-a230-369fdae10dcb\") " pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:23.087378 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.087319    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bprd9\" (UniqueName: \"kubernetes.io/projected/7cd72f17-5179-44c0-a06f-d330a2766e78-kube-api-access-bprd9\") pod \"node-ca-7287k\" (UID: \"7cd72f17-5179-44c0-a06f-d330a2766e78\") " pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:23.087442 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.087418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9vrl\" (UniqueName: \"kubernetes.io/projected/070d5e2b-0645-4ed5-89e5-d500952f8dbd-kube-api-access-q9vrl\") pod \"ovnkube-node-mllxt\" (UID: \"070d5e2b-0645-4ed5-89e5-d500952f8dbd\") " pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.087996 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.087976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv98s\" (UniqueName: \"kubernetes.io/projected/c47efd49-c2f8-4938-9a7b-7e32fc512951-kube-api-access-pv98s\") pod \"aws-ebs-csi-driver-node-gxtwg\" (UID: \"c47efd49-c2f8-4938-9a7b-7e32fc512951\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.088082 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.088047    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwzpj\" (UniqueName: \"kubernetes.io/projected/e8b6666b-4910-4bd4-a3a3-f1a086a67052-kube-api-access-jwzpj\") pod \"multus-ntq4l\" (UID: \"e8b6666b-4910-4bd4-a3a3-f1a086a67052\") " pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.088148 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.088105    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l26b\" (UniqueName: \"kubernetes.io/projected/d2bee516-e8ec-4bd5-bc9b-fde9685d9793-kube-api-access-2l26b\") pod \"tuned-xk96t\" (UID: \"d2bee516-e8ec-4bd5-bc9b-fde9685d9793\") " pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.088783 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.088768    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dxt9\" (UniqueName: \"kubernetes.io/projected/9f1c5761-f83c-4f1c-8994-8de155256444-kube-api-access-2dxt9\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:23.195720 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.195701    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-46t4x"
Sep 17 23:06:23.201473 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.201454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ntq4l"
Sep 17 23:06:23.212868 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.212845    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b6666b_4910_4bd4_a3a3_f1a086a67052.slice/crio-67a80e8476c1a399d1db97901bf415368386f98f0c244e1c53fa0759a9816875 WatchSource:0}: Error finding container 67a80e8476c1a399d1db97901bf415368386f98f0c244e1c53fa0759a9816875: Status 404 returned error can't find the container with id 67a80e8476c1a399d1db97901bf415368386f98f0c244e1c53fa0759a9816875
Sep 17 23:06:23.213406 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.213382    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1695e8f9_ec8b_41db_a230_369fdae10dcb.slice/crio-62e1450f62baba96bf33a1d55cd867f9d14b0a6ed2f5ab06fc1a39b83bf944f0 WatchSource:0}: Error finding container 62e1450f62baba96bf33a1d55cd867f9d14b0a6ed2f5ab06fc1a39b83bf944f0: Status 404 returned error can't find the container with id 62e1450f62baba96bf33a1d55cd867f9d14b0a6ed2f5ab06fc1a39b83bf944f0
Sep 17 23:06:23.214353 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.214325    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6989cb969ca76f3d903d482174d8453.slice/crio-5bf36b31c10a47f295309b44895bd8d73753cd428e96aee4f3b4d9b42e7a4188 WatchSource:0}: Error finding container 5bf36b31c10a47f295309b44895bd8d73753cd428e96aee4f3b4d9b42e7a4188: Status 404 returned error can't find the container with id 5bf36b31c10a47f295309b44895bd8d73753cd428e96aee4f3b4d9b42e7a4188
Sep 17 23:06:23.214866 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.214850    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5760489518c11caa7e8d239fc8d15015.slice/crio-66a10810547f7947fb9dcd97e5e179a9eed42bcb25ac0976253fbd73e2f3c6b1 WatchSource:0}: Error finding container 66a10810547f7947fb9dcd97e5e179a9eed42bcb25ac0976253fbd73e2f3c6b1: Status 404 returned error can't find the container with id 66a10810547f7947fb9dcd97e5e179a9eed42bcb25ac0976253fbd73e2f3c6b1
Sep 17 23:06:23.219223 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.219212    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:06:23.220934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.220918    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tl28n"
Sep 17 23:06:23.230238 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.230220    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:23.230497 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.230475    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f06477c_095c_4f4b_9dec_1ba898a0c47b.slice/crio-6972ace423a85d854b1a74ab04e43b2b9afe8940c07a35e1868144f74e768efc WatchSource:0}: Error finding container 6972ace423a85d854b1a74ab04e43b2b9afe8940c07a35e1868144f74e768efc: Status 404 returned error can't find the container with id 6972ace423a85d854b1a74ab04e43b2b9afe8940c07a35e1868144f74e768efc
Sep 17 23:06:23.239816 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.239796    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89970b05_164f_41fc_b269_0ae42eeddb40.slice/crio-5f9f8a56c2d36dedccfab121951b778715fe1e50492ce9aaf3a9bf4e5dcec74d WatchSource:0}: Error finding container 5f9f8a56c2d36dedccfab121951b778715fe1e50492ce9aaf3a9bf4e5dcec74d: Status 404 returned error can't find the container with id 5f9f8a56c2d36dedccfab121951b778715fe1e50492ce9aaf3a9bf4e5dcec74d
Sep 17 23:06:23.246929 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.246912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg"
Sep 17 23:06:23.254386 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.254365    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc47efd49_c2f8_4938_9a7b_7e32fc512951.slice/crio-e44994a5c9fad46b8586af07854d2e0267737d8c426ef0efe637635c588b875c WatchSource:0}: Error finding container e44994a5c9fad46b8586af07854d2e0267737d8c426ef0efe637635c588b875c: Status 404 returned error can't find the container with id e44994a5c9fad46b8586af07854d2e0267737d8c426ef0efe637635c588b875c
Sep 17 23:06:23.258227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.258209    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xk96t"
Sep 17 23:06:23.261893 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.261877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7287k"
Sep 17 23:06:23.265421 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.265405    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2bee516_e8ec_4bd5_bc9b_fde9685d9793.slice/crio-f8766c9d3dbca2abcdaa49431b7966bc898c120e22cd185daced1dad1cc272d6 WatchSource:0}: Error finding container f8766c9d3dbca2abcdaa49431b7966bc898c120e22cd185daced1dad1cc272d6: Status 404 returned error can't find the container with id f8766c9d3dbca2abcdaa49431b7966bc898c120e22cd185daced1dad1cc272d6
Sep 17 23:06:23.269640 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.269622    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd72f17_5179_44c0_a06f_d330a2766e78.slice/crio-2ba6234d9f15ce7cf87473cf93f86dabc74931d10e0adeb579f5f4ffe648d04f WatchSource:0}: Error finding container 2ba6234d9f15ce7cf87473cf93f86dabc74931d10e0adeb579f5f4ffe648d04f: Status 404 returned error can't find the container with id 2ba6234d9f15ce7cf87473cf93f86dabc74931d10e0adeb579f5f4ffe648d04f
Sep 17 23:06:23.280872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.280855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dbnmm"
Sep 17 23:06:23.285722 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.285708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:23.289689 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.289672    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1049d3e7_bedd_4ac4_b35d_f156a480d857.slice/crio-02c0ce7a7c46c24150d3c4653e854f1d4816bd60c945281f3a8bc614ac4a4bdf WatchSource:0}: Error finding container 02c0ce7a7c46c24150d3c4653e854f1d4816bd60c945281f3a8bc614ac4a4bdf: Status 404 returned error can't find the container with id 02c0ce7a7c46c24150d3c4653e854f1d4816bd60c945281f3a8bc614ac4a4bdf
Sep 17 23:06:23.293787 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:06:23.293767    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod070d5e2b_0645_4ed5_89e5_d500952f8dbd.slice/crio-03ff17518d6609e73e81c5c3719642e44ec8a7f323002647449668cd0d8bacd2 WatchSource:0}: Error finding container 03ff17518d6609e73e81c5c3719642e44ec8a7f323002647449668cd0d8bacd2: Status 404 returned error can't find the container with id 03ff17518d6609e73e81c5c3719642e44ec8a7f323002647449668cd0d8bacd2
Sep 17 23:06:23.295387 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.295368    2581 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-mllxt_openshift-ovn-kubernetes(070d5e2b-0645-4ed5-89e5-d500952f8dbd): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 17 23:06:23.296850 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.296832    2581 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-mllxt_openshift-ovn-kubernetes(070d5e2b-0645-4ed5-89e5-d500952f8dbd): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 17 23:06:23.299801 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.299760    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beacea521116b2422086941236135068766b54396a322e81fd4104d55c0282df\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" podUID="070d5e2b-0645-4ed5-89e5-d500952f8dbd"
Sep 17 23:06:23.436214 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.436187    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:23.479535 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.479462    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:23.479660 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.479575    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:23.479660 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.479625    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret podName:50b8ad4b-7ba8-4b82-9a42-6acdf46910b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:24.479611311 +0000 UTC m=+4.483803128 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret") pod "global-pull-secret-syncer-csnkx" (UID: "50b8ad4b-7ba8-4b82-9a42-6acdf46910b8") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:23.580684 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.580644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:23.580850 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.580780    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:23.580850 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.580847    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs podName:9f1c5761-f83c-4f1c-8994-8de155256444 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:24.580830535 +0000 UTC m=+4.585022356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs") pod "network-metrics-daemon-khgbr" (UID: "9f1c5761-f83c-4f1c-8994-8de155256444") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:23.681366 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.681329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:23.681540 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.681517    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:23.681540 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.681539    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:23.681643 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.681551    2581 projected.go:196] Error preparing data for projected volume kube-api-access-k6xv2 for pod openshift-network-diagnostics/network-check-target-xfshv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:23.681643 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:23.681615    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2 podName:89932788-5773-4bce-817e-a0041862d6a9 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:24.681594385 +0000 UTC m=+4.685786222 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6xv2" (UniqueName: "kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2") pod "network-check-target-xfshv" (UID: "89932788-5773-4bce-817e-a0041862d6a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:23.779438 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.779356    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:23.906162 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.905996    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 23:01:22 +0000 UTC" deadline="2028-04-13 13:24:21.052454675 +0000 UTC"
Sep 17 23:06:23.906162 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:23.906064    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13766h17m57.146394363s"
Sep 17 23:06:24.019249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.019190    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" event={"ID":"1049d3e7-bedd-4ac4-b35d-f156a480d857","Type":"ContainerStarted","Data":"02c0ce7a7c46c24150d3c4653e854f1d4816bd60c945281f3a8bc614ac4a4bdf"}
Sep 17 23:06:24.028346 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.028310    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7287k" event={"ID":"7cd72f17-5179-44c0-a06f-d330a2766e78","Type":"ContainerStarted","Data":"2ba6234d9f15ce7cf87473cf93f86dabc74931d10e0adeb579f5f4ffe648d04f"}
Sep 17 23:06:24.033375 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.033254    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xk96t" event={"ID":"d2bee516-e8ec-4bd5-bc9b-fde9685d9793","Type":"ContainerStarted","Data":"f8766c9d3dbca2abcdaa49431b7966bc898c120e22cd185daced1dad1cc272d6"}
Sep 17 23:06:24.039340 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.039311    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jq5sx" event={"ID":"89970b05-164f-41fc-b269-0ae42eeddb40","Type":"ContainerStarted","Data":"5f9f8a56c2d36dedccfab121951b778715fe1e50492ce9aaf3a9bf4e5dcec74d"}
Sep 17 23:06:24.047731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.047704    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tl28n" event={"ID":"0f06477c-095c-4f4b-9dec-1ba898a0c47b","Type":"ContainerStarted","Data":"6972ace423a85d854b1a74ab04e43b2b9afe8940c07a35e1868144f74e768efc"}
Sep 17 23:06:24.053946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.053920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-46t4x" event={"ID":"1695e8f9-ec8b-41db-a230-369fdae10dcb","Type":"ContainerStarted","Data":"62e1450f62baba96bf33a1d55cd867f9d14b0a6ed2f5ab06fc1a39b83bf944f0"}
Sep 17 23:06:24.062301 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.062271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal" event={"ID":"5760489518c11caa7e8d239fc8d15015","Type":"ContainerStarted","Data":"66a10810547f7947fb9dcd97e5e179a9eed42bcb25ac0976253fbd73e2f3c6b1"}
Sep 17 23:06:24.067515 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.067486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal" event={"ID":"e6989cb969ca76f3d903d482174d8453","Type":"ContainerStarted","Data":"5bf36b31c10a47f295309b44895bd8d73753cd428e96aee4f3b4d9b42e7a4188"}
Sep 17 23:06:24.077070 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.077041    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"03ff17518d6609e73e81c5c3719642e44ec8a7f323002647449668cd0d8bacd2"}
Sep 17 23:06:24.088548 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.088523    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg" event={"ID":"c47efd49-c2f8-4938-9a7b-7e32fc512951","Type":"ContainerStarted","Data":"e44994a5c9fad46b8586af07854d2e0267737d8c426ef0efe637635c588b875c"}
Sep 17 23:06:24.091214 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.091154    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ntq4l" event={"ID":"e8b6666b-4910-4bd4-a3a3-f1a086a67052","Type":"ContainerStarted","Data":"67a80e8476c1a399d1db97901bf415368386f98f0c244e1c53fa0759a9816875"}
Sep 17 23:06:24.107444 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.107364    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beacea521116b2422086941236135068766b54396a322e81fd4104d55c0282df\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:beacea521116b2422086941236135068766b54396a322e81fd4104d55c0282df\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e3a0df0e827e39567f587ece03a40630a77aaaeacecf070c175b10d086f0631e\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" podUID="070d5e2b-0645-4ed5-89e5-d500952f8dbd"
Sep 17 23:06:24.489299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.489260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:24.489484 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.489421    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:24.489484 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.489482    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret podName:50b8ad4b-7ba8-4b82-9a42-6acdf46910b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:26.489465275 +0000 UTC m=+6.493657096 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret") pod "global-pull-secret-syncer-csnkx" (UID: "50b8ad4b-7ba8-4b82-9a42-6acdf46910b8") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:24.590436 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.590351    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:24.590597 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.590515    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:24.590597 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.590579    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs podName:9f1c5761-f83c-4f1c-8994-8de155256444 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:26.590560945 +0000 UTC m=+6.594752777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs") pod "network-metrics-daemon-khgbr" (UID: "9f1c5761-f83c-4f1c-8994-8de155256444") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:24.692140 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.692091    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:24.692313 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.692270    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:24.692313 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.692289    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:24.692313 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.692301    2581 projected.go:196] Error preparing data for projected volume kube-api-access-k6xv2 for pod openshift-network-diagnostics/network-check-target-xfshv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:24.692477 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.692359    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2 podName:89932788-5773-4bce-817e-a0041862d6a9 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:26.692341524 +0000 UTC m=+6.696533357 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6xv2" (UniqueName: "kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2") pod "network-check-target-xfshv" (UID: "89932788-5773-4bce-817e-a0041862d6a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:24.906960 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.906914    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 23:01:22 +0000 UTC" deadline="2028-06-23 00:21:17.803874496 +0000 UTC"
Sep 17 23:06:24.906960 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.906960    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15457h14m52.89692172s"
Sep 17 23:06:24.987111 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.987077    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:24.987314 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.987137    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:24.987314 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.987222    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:24.987314 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:24.987294    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:24.987474 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.987389    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:24.987532 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:24.987472    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:25.249420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:25.249345    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:25.530014 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:25.529916    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 23:06:26.508910 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:26.508872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:26.509469 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.509029    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:26.509469 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.509109    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret podName:50b8ad4b-7ba8-4b82-9a42-6acdf46910b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:30.509084176 +0000 UTC m=+10.513276017 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret") pod "global-pull-secret-syncer-csnkx" (UID: "50b8ad4b-7ba8-4b82-9a42-6acdf46910b8") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:26.609338 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:26.609306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:26.609515 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.609465    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:26.609578 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.609532    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs podName:9f1c5761-f83c-4f1c-8994-8de155256444 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:30.609515021 +0000 UTC m=+10.613706854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs") pod "network-metrics-daemon-khgbr" (UID: "9f1c5761-f83c-4f1c-8994-8de155256444") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:26.710780 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:26.710734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:26.710947 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.710918    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:26.710947 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.710938    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:26.711078 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.710952    2581 projected.go:196] Error preparing data for projected volume kube-api-access-k6xv2 for pod openshift-network-diagnostics/network-check-target-xfshv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:26.711078 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.711019    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2 podName:89932788-5773-4bce-817e-a0041862d6a9 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:30.711001815 +0000 UTC m=+10.715193634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6xv2" (UniqueName: "kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2") pod "network-check-target-xfshv" (UID: "89932788-5773-4bce-817e-a0041862d6a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:26.987888 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:26.987853    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:26.988054 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.987985    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:26.988403 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:26.988383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:26.988542 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.988519    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:26.988585 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:26.988575    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:26.988658 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:26.988640    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:28.988070 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:28.988027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:28.988542 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:28.988181    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:28.988542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:28.988439    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:28.988683 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:28.988542    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:28.988683 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:28.988621    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:28.988788 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:28.988737    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:30.541763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:30.541668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:30.542213 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.541838    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:30.542213 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.541935    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret podName:50b8ad4b-7ba8-4b82-9a42-6acdf46910b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:38.541911561 +0000 UTC m=+18.546103384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret") pod "global-pull-secret-syncer-csnkx" (UID: "50b8ad4b-7ba8-4b82-9a42-6acdf46910b8") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:30.642063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:30.642021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:30.642267 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.642216    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:30.642329 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.642306    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs podName:9f1c5761-f83c-4f1c-8994-8de155256444 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:38.642283676 +0000 UTC m=+18.646475500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs") pod "network-metrics-daemon-khgbr" (UID: "9f1c5761-f83c-4f1c-8994-8de155256444") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:30.743738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:30.743168    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:30.743738 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.743340    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:30.743738 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.743360    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:30.743738 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.743371    2581 projected.go:196] Error preparing data for projected volume kube-api-access-k6xv2 for pod openshift-network-diagnostics/network-check-target-xfshv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:30.743738 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.743420    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2 podName:89932788-5773-4bce-817e-a0041862d6a9 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:38.743402331 +0000 UTC m=+18.747594196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6xv2" (UniqueName: "kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2") pod "network-check-target-xfshv" (UID: "89932788-5773-4bce-817e-a0041862d6a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:30.987903 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:30.987799    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:30.987903 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:30.987850    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:30.988170 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.987921    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:30.988231 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:30.988186    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:30.988341 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.988313    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:30.988440 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:30.988415    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:32.988073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:32.988029    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:32.988073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:32.988060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:32.988529 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:32.988058    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:32.988529 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:32.988178    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:32.988529 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:32.988321    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:32.988529 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:32.988385    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:34.987796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:34.987762    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:34.987796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:34.987780    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:34.988309 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:34.987765    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:34.988309 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:34.987890    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:34.988309 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:34.987946    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:34.988309 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:34.988015    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:36.987235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:36.987046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:36.987653 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:36.987096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:36.987653 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:36.987331    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:36.987653 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:36.987096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:36.987653 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:36.987396    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:36.987653 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:36.987431    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:37.117515 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:37.117484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xk96t" event={"ID":"d2bee516-e8ec-4bd5-bc9b-fde9685d9793","Type":"ContainerStarted","Data":"ea8c1712ba5559a3142fbcd84dd21329f95242ba7eecbe90d50ab61b189704e9"}
Sep 17 23:06:37.118967 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:37.118945    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal" event={"ID":"e6989cb969ca76f3d903d482174d8453","Type":"ContainerStarted","Data":"7ef11603c26dda4e0ca3d3ce1aaaba4464468b862005e67d41a9d0aee9daaa77"}
Sep 17 23:06:37.120318 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:37.120298    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ntq4l" event={"ID":"e8b6666b-4910-4bd4-a3a3-f1a086a67052","Type":"ContainerStarted","Data":"89900bef4c9a252367c62c7a352cb182b571e1736e070657f3c8113d2bafb59a"}
Sep 17 23:06:37.136815 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:37.136772    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xk96t" podStartSLOduration=2.20845463 podStartE2EDuration="15.136758273s" totalImagesPullingTime="12.928303643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:37.136203435 +0000 UTC m=+17.140395275" watchObservedRunningTime="2026-09-17 23:06:37.136758273 +0000 UTC m=+17.140950112"
Sep 17 23:06:37.157558 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:37.157522    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ntq4l" podStartSLOduration=2.148094896 podStartE2EDuration="15.157509456s" totalImagesPullingTime="13.00941456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:37.1571748 +0000 UTC m=+17.161366638" watchObservedRunningTime="2026-09-17 23:06:37.157509456 +0000 UTC m=+17.161701295"
Sep 17 23:06:37.170517 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:37.170476    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-233.ec2.internal" podStartSLOduration=15.170462454 podStartE2EDuration="15.170462454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:37.170302035 +0000 UTC m=+17.174493876" watchObservedRunningTime="2026-09-17 23:06:37.170462454 +0000 UTC m=+17.174654293"
Sep 17 23:06:38.119404 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.119380    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 23:06:38.123513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.123490    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-46t4x" event={"ID":"1695e8f9-ec8b-41db-a230-369fdae10dcb","Type":"ContainerStarted","Data":"3ca6669c27e5e535eb7a9669603f366712796ce9d487f8a6d9da1a7547062433"}
Sep 17 23:06:38.125024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.124999    2581 generic.go:356] "Generic (PLEG): container finished" podID="5760489518c11caa7e8d239fc8d15015" containerID="30e35880b1806cd708dc74cc28afceaf97f41144ba7781b20e79ffd6b7fb893f" exitCode=0
Sep 17 23:06:38.125103 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.125086    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal" event={"ID":"5760489518c11caa7e8d239fc8d15015","Type":"ContainerDied","Data":"30e35880b1806cd708dc74cc28afceaf97f41144ba7781b20e79ffd6b7fb893f"}
Sep 17 23:06:38.126941 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.126903    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg" event={"ID":"c47efd49-c2f8-4938-9a7b-7e32fc512951","Type":"ContainerStarted","Data":"95eb3b7be04fb19a716c682d9a6e002f5aed619c55bf2deabb559bc231d54744"}
Sep 17 23:06:38.127023 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.126973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg" event={"ID":"c47efd49-c2f8-4938-9a7b-7e32fc512951","Type":"ContainerStarted","Data":"b90b28ded43f3d125ece6131849be6d4f4306ae1b93611947461353af88cb870"}
Sep 17 23:06:38.128585 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.128564    2581 generic.go:356] "Generic (PLEG): container finished" podID="1049d3e7-bedd-4ac4-b35d-f156a480d857" containerID="d453218316b33eb411da2f84445eaaddef7c76d15f0504de77800bbea868cedf" exitCode=0
Sep 17 23:06:38.128680 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.128605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" event={"ID":"1049d3e7-bedd-4ac4-b35d-f156a480d857","Type":"ContainerDied","Data":"d453218316b33eb411da2f84445eaaddef7c76d15f0504de77800bbea868cedf"}
Sep 17 23:06:38.130296 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.130275    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7287k" event={"ID":"7cd72f17-5179-44c0-a06f-d330a2766e78","Type":"ContainerStarted","Data":"9fe003c3d512f574c4d49a1f1f2dabb4a3eed853773d05e91195fa7e749de75d"}
Sep 17 23:06:38.131767 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.131738    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jq5sx" event={"ID":"89970b05-164f-41fc-b269-0ae42eeddb40","Type":"ContainerStarted","Data":"1d63585bd96f59b43e4259784fd6b394200fb5d31f4866600b72481f28502961"}
Sep 17 23:06:38.133371 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.133347    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tl28n" event={"ID":"0f06477c-095c-4f4b-9dec-1ba898a0c47b","Type":"ContainerStarted","Data":"f68c7c25171b1be10b327b843939740cc7c2051c36a83ad4651fac6cef9592e2"}
Sep 17 23:06:38.138930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.138900    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-46t4x" podStartSLOduration=3.179386651 podStartE2EDuration="16.138888747s" totalImagesPullingTime="12.959502096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:38.138428355 +0000 UTC m=+18.142620200" watchObservedRunningTime="2026-09-17 23:06:38.138888747 +0000 UTC m=+18.143080585"
Sep 17 23:06:38.153506 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.153311    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jq5sx" podStartSLOduration=3.215029827 podStartE2EDuration="16.153294045s" totalImagesPullingTime="12.938264218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:38.15328462 +0000 UTC m=+18.157476460" watchObservedRunningTime="2026-09-17 23:06:38.153294045 +0000 UTC m=+18.157485886"
Sep 17 23:06:38.169193 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.169079    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7287k" podStartSLOduration=3.472946689 podStartE2EDuration="16.169062084s" totalImagesPullingTime="12.696115395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:38.168545624 +0000 UTC m=+18.172737466" watchObservedRunningTime="2026-09-17 23:06:38.169062084 +0000 UTC m=+18.173253928"
Sep 17 23:06:38.196632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.196576    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tl28n" podStartSLOduration=3.249522417 podStartE2EDuration="16.196559193s" totalImagesPullingTime="12.947036776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:38.196531787 +0000 UTC m=+18.200723618" watchObservedRunningTime="2026-09-17 23:06:38.196559193 +0000 UTC m=+18.200751033"
Sep 17 23:06:38.333652 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.333562    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:38.417958 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.417928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:38.518773 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.518734    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:38.603560 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.601516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:38.603560 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.601748    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:38.603560 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.601847    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret podName:50b8ad4b-7ba8-4b82-9a42-6acdf46910b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:54.601820042 +0000 UTC m=+34.606011862 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret") pod "global-pull-secret-syncer-csnkx" (UID: "50b8ad4b-7ba8-4b82-9a42-6acdf46910b8") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:38.618333 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.618299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:38.702347 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.702312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:38.702503 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.702465    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:38.702567 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.702536    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs podName:9f1c5761-f83c-4f1c-8994-8de155256444 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:54.702517283 +0000 UTC m=+34.706709117 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs") pod "network-metrics-daemon-khgbr" (UID: "9f1c5761-f83c-4f1c-8994-8de155256444") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:38.802791 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.802750    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:38.803015 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.802918    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:38.803015 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.802941    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:38.803015 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.802954    2581 projected.go:196] Error preparing data for projected volume kube-api-access-k6xv2 for pod openshift-network-diagnostics/network-check-target-xfshv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:38.803015 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.803015    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2 podName:89932788-5773-4bce-817e-a0041862d6a9 nodeName:}" failed. No retries permitted until 2026-09-17 23:06:54.802997261 +0000 UTC m=+34.807189086 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6xv2" (UniqueName: "kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2") pod "network-check-target-xfshv" (UID: "89932788-5773-4bce-817e-a0041862d6a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:38.961776 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.961546    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T23:06:38.119399676Z","UUID":"17c638f0-e767-48e5-a9ac-629538250a27","Handler":null,"Name":"","Endpoint":""}
Sep 17 23:06:38.963266 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.963247    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 23:06:38.963404 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.963279    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 23:06:38.987825 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.987779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:38.987825 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.987811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:38.988112 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.987940    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:38.988112 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.988063    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:38.988286 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:38.988213    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:38.988366 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:38.988332    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:39.138175 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:39.138117    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal" event={"ID":"5760489518c11caa7e8d239fc8d15015","Type":"ContainerStarted","Data":"ab80d53cf9be7ea2cdeb1ab2ce4b9d80c683c102f3abc0c8cd37c1e993006fc3"}
Sep 17 23:06:39.140678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:39.140648    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg" event={"ID":"c47efd49-c2f8-4938-9a7b-7e32fc512951","Type":"ContainerStarted","Data":"12cdd0d34ce379aa995775a64c44c98ce3617275029b9eb24cbf53ec14e5a20b"}
Sep 17 23:06:39.169200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:39.169100    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gxtwg" podStartSLOduration=2.733784938 podStartE2EDuration="17.169087029s" totalImagesPullingTime="14.435302091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:39.169022256 +0000 UTC m=+19.173214096" watchObservedRunningTime="2026-09-17 23:06:39.169087029 +0000 UTC m=+19.173278898"
Sep 17 23:06:39.169560 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:39.169530    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-233.ec2.internal" podStartSLOduration=17.169524346 podStartE2EDuration="17.169524346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:39.152781344 +0000 UTC m=+19.156973186" watchObservedRunningTime="2026-09-17 23:06:39.169524346 +0000 UTC m=+19.173716192"
Sep 17 23:06:40.205586 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:40.205533    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:40.305607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:40.305575    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:40.987843 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:40.987748    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:40.987843 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:40.987800    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:40.988091 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:40.987881    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:40.988091 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:40.987890    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:40.988091 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:40.987982    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:40.988091 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:40.988069    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:42.117260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:42.117225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:42.205901 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:42.205866    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:42.728709 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:42.728672    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:42.729541 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:42.729518    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:42.987242 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:42.987173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:42.987242 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:42.987191    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:42.987242 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:42.987207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:42.987488 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:42.987281    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:42.987488 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:42.987366    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:42.987488 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:42.987463    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:43.149950 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:43.149918    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:43.150480 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:43.150391    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jq5sx"
Sep 17 23:06:43.212529 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:43.212497    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:44.154246 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:44.154203    2581 generic.go:356] "Generic (PLEG): container finished" podID="1049d3e7-bedd-4ac4-b35d-f156a480d857" containerID="d78a10390e2bf3b21c61eed430e9ef626e53ad343aae8db26d9c703f4d57b851" exitCode=0
Sep 17 23:06:44.154686 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:44.154271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" event={"ID":"1049d3e7-bedd-4ac4-b35d-f156a480d857","Type":"ContainerDied","Data":"d78a10390e2bf3b21c61eed430e9ef626e53ad343aae8db26d9c703f4d57b851"}
Sep 17 23:06:44.987718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:44.987481    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:44.987904 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:44.987586    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:44.987904 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:44.987839    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:44.987904 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:44.987621    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:44.987904 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:44.987870    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:44.988109 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:44.987933    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:46.987867 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:46.987835    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:46.987867 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:46.987855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:46.988399 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:46.987838    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:46.988399 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:46.987971    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:46.988399 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:46.988060    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:46.988399 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:46.988160    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:47.385264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:47.385228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:47.470667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:47.470635    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:47.544285 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:47.544252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:47.610872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:47.610832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:47.671003 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:47.670930    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:47.732108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:47.732070    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log" limit=1024
Sep 17 23:06:48.988188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:48.988019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:48.988487 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:48.988019    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:48.988487 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:48.988049    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:48.988487 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:48.988272    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:48.988487 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:48.988356    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:48.988487 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:48.988420    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:49.166785 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:49.166749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"30a4358c10d8935ce4669f6b7cdf9b7c7527cfa5f6349c1dad1505ee54a0ad33"}
Sep 17 23:06:49.167054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:49.166833    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"5b07aae0b29a10cb4d20751ce49873d50c57857d3af9ce7da1e845fbae1dc06b"}
Sep 17 23:06:49.167054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:49.166887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"7753d130e5e5d8d6e09fc6ec0b23317043b6b8b85d7587ce11092c85ec9b1f5b"}
Sep 17 23:06:49.167054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:49.166919    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"aebf31bf12d8de3b8eeb983168affeb68e044c38c704e3427144c78129de6acb"}
Sep 17 23:06:49.167054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:49.166959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"b85d686f9d40d3294f1325270f0bdcd7d587d5fb8d3eae0ce66a8d341e10cfe3"}
Sep 17 23:06:49.167054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:49.166990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"8fce5b103152ca9ea71b8d3612fdc5c4becc5f0bb9006ad72e0a5d2c4580294c"}
Sep 17 23:06:49.168946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:49.168916    2581 generic.go:356] "Generic (PLEG): container finished" podID="1049d3e7-bedd-4ac4-b35d-f156a480d857" containerID="66092bdf06082ca331f264c6546a0dee0416db95ade995d1a4d4a8616e9bb05a" exitCode=0
Sep 17 23:06:49.169044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:49.168968    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" event={"ID":"1049d3e7-bedd-4ac4-b35d-f156a480d857","Type":"ContainerDied","Data":"66092bdf06082ca331f264c6546a0dee0416db95ade995d1a4d4a8616e9bb05a"}
Sep 17 23:06:50.987089 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:50.987046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:50.987456 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:50.987097    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:50.987456 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:50.987160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:50.987456 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:50.987177    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:50.987456 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:50.987238    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:50.987456 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:50.987311    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:51.175990 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:51.175966    2581 generic.go:356] "Generic (PLEG): container finished" podID="1049d3e7-bedd-4ac4-b35d-f156a480d857" containerID="a47323674008385fa6bba4db7cd2bee4671c92651541dbbc7fe63d26f6bfffa7" exitCode=0
Sep 17 23:06:51.176095 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:51.176004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" event={"ID":"1049d3e7-bedd-4ac4-b35d-f156a480d857","Type":"ContainerDied","Data":"a47323674008385fa6bba4db7cd2bee4671c92651541dbbc7fe63d26f6bfffa7"}
Sep 17 23:06:52.182445 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:52.182408    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"92c57ee64c1455c8a25df7bbd1cad862ea373bb4b116111116df95d4b7f84b2b"}
Sep 17 23:06:52.987609 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:52.987571    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:52.987609 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:52.987596    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:52.987609 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:52.987609    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:52.987895 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:52.987691    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:52.987895 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:52.987805    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:52.987895 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:52.987885    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:54.192360 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.192152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" event={"ID":"070d5e2b-0645-4ed5-89e5-d500952f8dbd","Type":"ContainerStarted","Data":"2882790ff3cd5df332ddecb35eca1dbde248c42c72067f8926ed5f198b1c606d"}
Sep 17 23:06:54.193169 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.192434    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:54.193169 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.192493    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:54.209578 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.209559    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:54.234800 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.234749    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt" podStartSLOduration=7.339974981 podStartE2EDuration="32.234735065s" totalImagesPullingTime="24.894760084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-09-17 23:06:54.234633193 +0000 UTC m=+34.238825070" watchObservedRunningTime="2026-09-17 23:06:54.234735065 +0000 UTC m=+34.238926915"
Sep 17 23:06:54.621502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.618678    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:54.621502 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.618866    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:54.621502 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.618932    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret podName:50b8ad4b-7ba8-4b82-9a42-6acdf46910b8 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:26.618913177 +0000 UTC m=+66.623104994 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret") pod "global-pull-secret-syncer-csnkx" (UID: "50b8ad4b-7ba8-4b82-9a42-6acdf46910b8") : object "kube-system"/"original-pull-secret" not registered
Sep 17 23:06:54.719643 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.719602    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:54.719833 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.719744    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:54.719833 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.719809    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs podName:9f1c5761-f83c-4f1c-8994-8de155256444 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:26.719790976 +0000 UTC m=+66.723982810 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs") pod "network-metrics-daemon-khgbr" (UID: "9f1c5761-f83c-4f1c-8994-8de155256444") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 23:06:54.820690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.820641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:54.820863 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.820805    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 23:06:54.820863 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.820829    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 23:06:54.820863 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.820840    2581 projected.go:196] Error preparing data for projected volume kube-api-access-k6xv2 for pod openshift-network-diagnostics/network-check-target-xfshv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:54.821021 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.820905    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2 podName:89932788-5773-4bce-817e-a0041862d6a9 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:26.820887339 +0000 UTC m=+66.825079175 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k6xv2" (UniqueName: "kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2") pod "network-check-target-xfshv" (UID: "89932788-5773-4bce-817e-a0041862d6a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 23:06:54.987923 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.987843    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:54.987923 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.987862    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:54.988174 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.987967    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:54.988174 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:54.987971    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:54.988174 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.988055    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:54.988174 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:54.988103    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:55.196119 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:55.196091    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:55.215919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:55.215897    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:06:55.437568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:55.437540    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-khgbr"]
Sep 17 23:06:55.437690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:55.437682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:55.437802 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:55.437784    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:55.441297 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:55.441175    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-csnkx"]
Sep 17 23:06:55.441412 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:55.441345    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xfshv"]
Sep 17 23:06:55.441470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:55.441431    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:55.441470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:55.441460    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:55.441571 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:55.441526    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:55.441645 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:55.441624    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:56.200279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:56.200249    2581 generic.go:356] "Generic (PLEG): container finished" podID="1049d3e7-bedd-4ac4-b35d-f156a480d857" containerID="8fcc2e504d489e2a48d98498dd9384c0e472c2a1a7641110d6d85793206ecfaf" exitCode=0
Sep 17 23:06:56.200739 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:56.200317    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" event={"ID":"1049d3e7-bedd-4ac4-b35d-f156a480d857","Type":"ContainerDied","Data":"8fcc2e504d489e2a48d98498dd9384c0e472c2a1a7641110d6d85793206ecfaf"}
Sep 17 23:06:56.212348 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:56.212329    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-46t4x_1695e8f9-ec8b-41db-a230-369fdae10dcb/dns-node-resolver/0.log"
Sep 17 23:06:56.987906 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:56.987864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:56.988097 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:56.987864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:56.988097 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:56.987997    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:56.988097 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:56.987864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:56.988097 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:56.988065    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:56.988342 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:56.988167    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:06:57.205206 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:57.205170    2581 generic.go:356] "Generic (PLEG): container finished" podID="1049d3e7-bedd-4ac4-b35d-f156a480d857" containerID="bd2cdc64bba84ffb219f6a072569379b845c0f2f90847f4058b0bd4697622a1f" exitCode=0
Sep 17 23:06:57.205570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:57.205249    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" event={"ID":"1049d3e7-bedd-4ac4-b35d-f156a480d857","Type":"ContainerDied","Data":"bd2cdc64bba84ffb219f6a072569379b845c0f2f90847f4058b0bd4697622a1f"}
Sep 17 23:06:57.209421 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:57.209402    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7287k_7cd72f17-5179-44c0-a06f-d330a2766e78/node-ca/0.log"
Sep 17 23:06:58.213481 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:58.213311    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" event={"ID":"1049d3e7-bedd-4ac4-b35d-f156a480d857","Type":"ContainerStarted","Data":"c7d3caf42261da293cdb72a1392e3bbcfde37f0dee9cf208d9b75206a3454cd2"}
Sep 17 23:06:58.237162 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:58.237098    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dbnmm" podStartSLOduration=8.452126949 podStartE2EDuration="36.237084076s" totalImagesPullingTime="27.784957127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:06:58.236662863 +0000 UTC m=+38.240854701" watchObservedRunningTime="2026-09-17 23:06:58.237084076 +0000 UTC m=+38.241275914"
Sep 17 23:06:58.987619 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:58.987584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:06:58.987801 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:58.987637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:06:58.987801 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:06:58.987663    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:06:58.987801 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:58.987749    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:06:58.987801 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:58.987797    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:06:58.987942 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:06:58.987852    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:00.987239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:00.987210    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:00.987679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:00.987207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:00.987679 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:00.987337    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:00.987679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:00.987212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:00.987679 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:00.987442    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:00.987679 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:00.987381    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:02.987013 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:02.986974    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:02.987013 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:02.986997    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:02.987521 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:02.986975    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:02.987521 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:02.987077    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:02.987521 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:02.987211    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:02.987521 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:02.987296    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:04.987982 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:04.987956    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:04.988302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:04.987984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:04.988302 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:04.988042    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:04.988302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:04.988097    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:04.988302 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:04.988215    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:04.988302 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:04.988294    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:06.988063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:06.988027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:06.988618 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:06.988069    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:06.988618 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:06.988134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:06.988618 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:06.988159    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:06.988618 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:06.988211    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:06.988618 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:06.988262    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:08.987960 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:08.987933    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:08.987960 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:08.987948    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:08.988552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:08.987935    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:08.988552 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:08.988039    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:08.988552 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:08.988107    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:08.988552 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:08.988205    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:10.987347 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:10.987314    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:10.987347 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:10.987342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:10.987969 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:10.987405    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:10.987969 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:10.987506    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:10.987969 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:10.987599    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:10.987969 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:10.987667    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:12.987662 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:12.987628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:12.988024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:12.987632    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:12.988024 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:12.987731    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:12.988024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:12.987632    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:12.988024 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:12.987804    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:12.988024 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:12.987873    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:14.987740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:14.987694    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:14.988161 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:14.987715    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:14.988161 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:14.987800    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:14.988161 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:14.987896    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:14.988161 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:14.987695    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:14.988161 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:14.987976    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:16.987523 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:16.987493    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:16.987884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:16.987608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:16.987884 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:16.987620    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:16.987884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:16.987635    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:16.987884 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:16.987691    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:16.987884 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:16.987752    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:18.987444 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:18.987409    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:18.987444 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:18.987438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:18.987835 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:18.987526    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:18.987835 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:18.987559    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:18.987835 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:18.987613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:18.987835 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:18.987711    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:20.987078 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:20.987043    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:20.987563 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:20.987120    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:20.987563 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:20.987148    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xfshv" podUID="89932788-5773-4bce-817e-a0041862d6a9"
Sep 17 23:07:20.987563 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:20.987174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:20.987563 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:20.987282    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-khgbr" podUID="9f1c5761-f83c-4f1c-8994-8de155256444"
Sep 17 23:07:20.987563 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:20.987319    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-csnkx" podUID="50b8ad4b-7ba8-4b82-9a42-6acdf46910b8"
Sep 17 23:07:22.567597 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.567569    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-233.ec2.internal" event="NodeReady"
Sep 17 23:07:22.568073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.567678    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 23:07:22.607715 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.607687    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7646669d8b-fwlsv"]
Sep 17 23:07:22.632606 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.632580    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7646669d8b-fwlsv"]
Sep 17 23:07:22.632606 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.632609    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9sscn"]
Sep 17 23:07:22.632769 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.632691    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.635453 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.635428    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 23:07:22.635598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.635488    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 23:07:22.635747 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.635731    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dwfw7\""
Sep 17 23:07:22.636177 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.636161    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 23:07:22.642021 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.641999    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 23:07:22.643654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.643634    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tk7qr"]
Sep 17 23:07:22.643797 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.643779    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.646571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.646548    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 23:07:22.646702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.646684    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 23:07:22.646922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.646908    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-62qw9\""
Sep 17 23:07:22.659762 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.659742    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9sscn"]
Sep 17 23:07:22.659849 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.659779    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tk7qr"]
Sep 17 23:07:22.659887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.659864    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.663405 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.663386    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 23:07:22.663523 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.663410    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 23:07:22.663523 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.663388    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 23:07:22.663523 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.663387    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 23:07:22.663724 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.663709    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bbrzm\""
Sep 17 23:07:22.728951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.728924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd708c1e-4ada-427c-9d9e-fd47a0980689-trusted-ca\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.728951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.728955    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/227b9f31-6478-4430-ae85-fa6085c296cd-config-volume\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.729200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd708c1e-4ada-427c-9d9e-fd47a0980689-installation-pull-secrets\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.729200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729033    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bhv4\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-kube-api-access-9bhv4\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.729200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/227b9f31-6478-4430-ae85-fa6085c296cd-tmp-dir\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.729200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729095    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/227b9f31-6478-4430-ae85-fa6085c296cd-metrics-tls\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.729200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-registry-tls\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.729424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729199    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd708c1e-4ada-427c-9d9e-fd47a0980689-registry-certificates\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.729424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac663cea-8971-4476-b0ef-fc259d52500f-data-volume\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.729424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq7d7\" (UniqueName: \"kubernetes.io/projected/ac663cea-8971-4476-b0ef-fc259d52500f-kube-api-access-sq7d7\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.729424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729304    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-bound-sa-token\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.729424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729354    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd2jt\" (UniqueName: \"kubernetes.io/projected/227b9f31-6478-4430-ae85-fa6085c296cd-kube-api-access-cd2jt\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.729424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729387    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd708c1e-4ada-427c-9d9e-fd47a0980689-image-registry-private-configuration\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.729424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729405    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd708c1e-4ada-427c-9d9e-fd47a0980689-ca-trust-extracted\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.729789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac663cea-8971-4476-b0ef-fc259d52500f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.729789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac663cea-8971-4476-b0ef-fc259d52500f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.729789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.729537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ac663cea-8971-4476-b0ef-fc259d52500f-crio-socket\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.730668 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.730650    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x7bqz"]
Sep 17 23:07:22.751220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.751198    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x7bqz"]
Sep 17 23:07:22.751340 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.751276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x7bqz"
Sep 17 23:07:22.754785 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.754763    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 23:07:22.754897 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.754766    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ljrmd\""
Sep 17 23:07:22.754897 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.754800    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 23:07:22.754897 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.754807    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 23:07:22.803038 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.802995    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56fc9b5857-5v5zd"]
Sep 17 23:07:22.823218 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.823167    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:22.825105 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.824805    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56fc9b5857-5v5zd"]
Sep 17 23:07:22.827145 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.826807    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 23:07:22.828826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.827766    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 23:07:22.828826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.828144    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 23:07:22.828826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.828464    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-t72j2\""
Sep 17 23:07:22.828826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.828535    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 23:07:22.828826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.828463    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 23:07:22.828826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.828769    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 23:07:22.829188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.828983    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 23:07:22.829862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.829837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/227b9f31-6478-4430-ae85-fa6085c296cd-metrics-tls\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.829969 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.829900    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-registry-tls\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.829969 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.829928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd708c1e-4ada-427c-9d9e-fd47a0980689-registry-certificates\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.829969 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.829953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac663cea-8971-4476-b0ef-fc259d52500f-data-volume\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.830150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.829978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq7d7\" (UniqueName: \"kubernetes.io/projected/ac663cea-8971-4476-b0ef-fc259d52500f-kube-api-access-sq7d7\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.830150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830008    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-bound-sa-token\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.830386 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd2jt\" (UniqueName: \"kubernetes.io/projected/227b9f31-6478-4430-ae85-fa6085c296cd-kube-api-access-cd2jt\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.830476 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd708c1e-4ada-427c-9d9e-fd47a0980689-image-registry-private-configuration\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.830536 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830486    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac663cea-8971-4476-b0ef-fc259d52500f-data-volume\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.830682 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd708c1e-4ada-427c-9d9e-fd47a0980689-ca-trust-extracted\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.830765 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830700    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9sl\" (UniqueName: \"kubernetes.io/projected/9bdaf271-bcf3-4c34-865f-4ec53f8d5c37-kube-api-access-2l9sl\") pod \"ingress-canary-x7bqz\" (UID: \"9bdaf271-bcf3-4c34-865f-4ec53f8d5c37\") " pod="openshift-ingress-canary/ingress-canary-x7bqz"
Sep 17 23:07:22.830765 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac663cea-8971-4476-b0ef-fc259d52500f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.830859 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac663cea-8971-4476-b0ef-fc259d52500f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.830859 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ac663cea-8971-4476-b0ef-fc259d52500f-crio-socket\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.831014 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd708c1e-4ada-427c-9d9e-fd47a0980689-trusted-ca\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.831014 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/227b9f31-6478-4430-ae85-fa6085c296cd-config-volume\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.831014 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bdaf271-bcf3-4c34-865f-4ec53f8d5c37-cert\") pod \"ingress-canary-x7bqz\" (UID: \"9bdaf271-bcf3-4c34-865f-4ec53f8d5c37\") " pod="openshift-ingress-canary/ingress-canary-x7bqz"
Sep 17 23:07:22.831014 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.830992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd708c1e-4ada-427c-9d9e-fd47a0980689-installation-pull-secrets\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.831014 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.831003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fd708c1e-4ada-427c-9d9e-fd47a0980689-registry-certificates\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.831014 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.831013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bhv4\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-kube-api-access-9bhv4\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.831329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.831034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/227b9f31-6478-4430-ae85-fa6085c296cd-tmp-dir\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.831329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.831065    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fd708c1e-4ada-427c-9d9e-fd47a0980689-ca-trust-extracted\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.831425 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.831370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/227b9f31-6478-4430-ae85-fa6085c296cd-tmp-dir\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.831750 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.831713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ac663cea-8971-4476-b0ef-fc259d52500f-crio-socket\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.831862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.831844    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/227b9f31-6478-4430-ae85-fa6085c296cd-config-volume\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.832021 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.831994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ac663cea-8971-4476-b0ef-fc259d52500f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.832197 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.832172    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fd708c1e-4ada-427c-9d9e-fd47a0980689-trusted-ca\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.834433 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.834408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/227b9f31-6478-4430-ae85-fa6085c296cd-metrics-tls\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.834604 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.834581    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fd708c1e-4ada-427c-9d9e-fd47a0980689-image-registry-private-configuration\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.834695 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.834667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fd708c1e-4ada-427c-9d9e-fd47a0980689-installation-pull-secrets\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.834902 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.834878    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-registry-tls\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.834999 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.834981    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ac663cea-8971-4476-b0ef-fc259d52500f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.838993 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.838969    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-bound-sa-token\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.839148 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.839115    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq7d7\" (UniqueName: \"kubernetes.io/projected/ac663cea-8971-4476-b0ef-fc259d52500f-kube-api-access-sq7d7\") pod \"insights-runtime-extractor-tk7qr\" (UID: \"ac663cea-8971-4476-b0ef-fc259d52500f\") " pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.843625 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.843603    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bhv4\" (UniqueName: \"kubernetes.io/projected/fd708c1e-4ada-427c-9d9e-fd47a0980689-kube-api-access-9bhv4\") pod \"image-registry-7646669d8b-fwlsv\" (UID: \"fd708c1e-4ada-427c-9d9e-fd47a0980689\") " pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.845000 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.844984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd2jt\" (UniqueName: \"kubernetes.io/projected/227b9f31-6478-4430-ae85-fa6085c296cd-kube-api-access-cd2jt\") pod \"dns-default-9sscn\" (UID: \"227b9f31-6478-4430-ae85-fa6085c296cd\") " pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.932083 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.932043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9sl\" (UniqueName: \"kubernetes.io/projected/9bdaf271-bcf3-4c34-865f-4ec53f8d5c37-kube-api-access-2l9sl\") pod \"ingress-canary-x7bqz\" (UID: \"9bdaf271-bcf3-4c34-865f-4ec53f8d5c37\") " pod="openshift-ingress-canary/ingress-canary-x7bqz"
Sep 17 23:07:22.932247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.932093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9p9x\" (UniqueName: \"kubernetes.io/projected/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-kube-api-access-t9p9x\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:22.932247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.932138    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-config\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:22.932247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.932161    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-service-ca\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:22.932247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.932187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bdaf271-bcf3-4c34-865f-4ec53f8d5c37-cert\") pod \"ingress-canary-x7bqz\" (UID: \"9bdaf271-bcf3-4c34-865f-4ec53f8d5c37\") " pod="openshift-ingress-canary/ingress-canary-x7bqz"
Sep 17 23:07:22.932394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.932324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-serving-cert\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:22.932394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.932356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-oauth-config\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:22.932394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.932385    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-oauth-serving-cert\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:22.934517 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.934495    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bdaf271-bcf3-4c34-865f-4ec53f8d5c37-cert\") pod \"ingress-canary-x7bqz\" (UID: \"9bdaf271-bcf3-4c34-865f-4ec53f8d5c37\") " pod="openshift-ingress-canary/ingress-canary-x7bqz"
Sep 17 23:07:22.941513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.941494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:22.950232 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.950212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9sl\" (UniqueName: \"kubernetes.io/projected/9bdaf271-bcf3-4c34-865f-4ec53f8d5c37-kube-api-access-2l9sl\") pod \"ingress-canary-x7bqz\" (UID: \"9bdaf271-bcf3-4c34-865f-4ec53f8d5c37\") " pod="openshift-ingress-canary/ingress-canary-x7bqz"
Sep 17 23:07:22.952006 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.951975    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:22.968149 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.968115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tk7qr"
Sep 17 23:07:22.987577 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.987562    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:22.987700 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.987682    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:22.988020 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.987826    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:22.990547 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.990530    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 23:07:22.990654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.990547    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 23:07:22.990654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.990578    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8cnd\""
Sep 17 23:07:22.990654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.990588    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 23:07:22.990907 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.990889    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 23:07:22.990983 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:22.990915    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fkqjx\""
Sep 17 23:07:23.034093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.032844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-serving-cert\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.034093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.032889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-oauth-config\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.034093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.032918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-oauth-serving-cert\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.034093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.032964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9p9x\" (UniqueName: \"kubernetes.io/projected/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-kube-api-access-t9p9x\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.034093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.032992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-config\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.034093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.033017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-service-ca\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.034093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.033852    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-service-ca\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.034093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.034035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-oauth-serving-cert\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.034600 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.034220    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-config\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.035792 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.035739    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-serving-cert\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.036083 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.036063    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-oauth-config\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.043991 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.043949    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9p9x\" (UniqueName: \"kubernetes.io/projected/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-kube-api-access-t9p9x\") pod \"console-56fc9b5857-5v5zd\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") " pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.060287 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.060242    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x7bqz"
Sep 17 23:07:23.153782 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.153759    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:23.286704 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.286669    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9sscn"]
Sep 17 23:07:23.289020 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.288993    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7646669d8b-fwlsv"]
Sep 17 23:07:23.291319 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:23.291296    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod227b9f31_6478_4430_ae85_fa6085c296cd.slice/crio-15802f832b40e67ebbd23320d9350e48098c6508b5e1378b1dc84efd46619e4e WatchSource:0}: Error finding container 15802f832b40e67ebbd23320d9350e48098c6508b5e1378b1dc84efd46619e4e: Status 404 returned error can't find the container with id 15802f832b40e67ebbd23320d9350e48098c6508b5e1378b1dc84efd46619e4e
Sep 17 23:07:23.292155 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:23.292116    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd708c1e_4ada_427c_9d9e_fd47a0980689.slice/crio-75a89df95126f9d5d516beb37ac588bd07e83ae18de42a8aaf77b0b4f238e62b WatchSource:0}: Error finding container 75a89df95126f9d5d516beb37ac588bd07e83ae18de42a8aaf77b0b4f238e62b: Status 404 returned error can't find the container with id 75a89df95126f9d5d516beb37ac588bd07e83ae18de42a8aaf77b0b4f238e62b
Sep 17 23:07:23.296146 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.296106    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tk7qr"]
Sep 17 23:07:23.298558 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:23.298535    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac663cea_8971_4476_b0ef_fc259d52500f.slice/crio-5281df53aaa15dc24c745967ab1e0b7d712548d7294e9c619883148e1baedb9a WatchSource:0}: Error finding container 5281df53aaa15dc24c745967ab1e0b7d712548d7294e9c619883148e1baedb9a: Status 404 returned error can't find the container with id 5281df53aaa15dc24c745967ab1e0b7d712548d7294e9c619883148e1baedb9a
Sep 17 23:07:23.357542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.357519    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x7bqz"]
Sep 17 23:07:23.376458 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:23.376435    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdaf271_bcf3_4c34_865f_4ec53f8d5c37.slice/crio-f45df1f00d0799ad248e35b903dd9cc35066c5137ff73d3b3e96574ae4149b36 WatchSource:0}: Error finding container f45df1f00d0799ad248e35b903dd9cc35066c5137ff73d3b3e96574ae4149b36: Status 404 returned error can't find the container with id f45df1f00d0799ad248e35b903dd9cc35066c5137ff73d3b3e96574ae4149b36
Sep 17 23:07:23.424679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:23.424647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56fc9b5857-5v5zd"]
Sep 17 23:07:23.427580 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:23.427552    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f4233bc_3012_4a93_9e5d_a0a07a57b8e3.slice/crio-72226ada883d8fd68b3a4bd29d84f0c034d366fe49fde956e16d6c2c6054edea WatchSource:0}: Error finding container 72226ada883d8fd68b3a4bd29d84f0c034d366fe49fde956e16d6c2c6054edea: Status 404 returned error can't find the container with id 72226ada883d8fd68b3a4bd29d84f0c034d366fe49fde956e16d6c2c6054edea
Sep 17 23:07:24.272746 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.272674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fc9b5857-5v5zd" event={"ID":"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3","Type":"ContainerStarted","Data":"72226ada883d8fd68b3a4bd29d84f0c034d366fe49fde956e16d6c2c6054edea"}
Sep 17 23:07:24.274772 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.274718    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x7bqz" event={"ID":"9bdaf271-bcf3-4c34-865f-4ec53f8d5c37","Type":"ContainerStarted","Data":"f45df1f00d0799ad248e35b903dd9cc35066c5137ff73d3b3e96574ae4149b36"}
Sep 17 23:07:24.278690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.278585    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tk7qr" event={"ID":"ac663cea-8971-4476-b0ef-fc259d52500f","Type":"ContainerStarted","Data":"f4ec3f5302761eaf9511b54b38b86559ca36e01e0d11b8574cf98a2919d4d5f6"}
Sep 17 23:07:24.278690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.278636    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tk7qr" event={"ID":"ac663cea-8971-4476-b0ef-fc259d52500f","Type":"ContainerStarted","Data":"5281df53aaa15dc24c745967ab1e0b7d712548d7294e9c619883148e1baedb9a"}
Sep 17 23:07:24.281462 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.281383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9sscn" event={"ID":"227b9f31-6478-4430-ae85-fa6085c296cd","Type":"ContainerStarted","Data":"15802f832b40e67ebbd23320d9350e48098c6508b5e1378b1dc84efd46619e4e"}
Sep 17 23:07:24.285968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.285176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7646669d8b-fwlsv" event={"ID":"fd708c1e-4ada-427c-9d9e-fd47a0980689","Type":"ContainerStarted","Data":"ed7f84ab68fcd986949220ac783e04abe34c21112db0e5b696ff9eba7c807ba6"}
Sep 17 23:07:24.285968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.285217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7646669d8b-fwlsv" event={"ID":"fd708c1e-4ada-427c-9d9e-fd47a0980689","Type":"ContainerStarted","Data":"75a89df95126f9d5d516beb37ac588bd07e83ae18de42a8aaf77b0b4f238e62b"}
Sep 17 23:07:24.285968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.285919    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:24.308422 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:24.308364    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7646669d8b-fwlsv" podStartSLOduration=3.308345602 podStartE2EDuration="3.308345602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:24.306916204 +0000 UTC m=+64.311108045" watchObservedRunningTime="2026-09-17 23:07:24.308345602 +0000 UTC m=+64.312537444"
Sep 17 23:07:26.664582 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.664541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:26.667054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.667025    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50b8ad4b-7ba8-4b82-9a42-6acdf46910b8-original-pull-secret\") pod \"global-pull-secret-syncer-csnkx\" (UID: \"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8\") " pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:26.765333 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.765294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:26.767724 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.767694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f1c5761-f83c-4f1c-8994-8de155256444-metrics-certs\") pod \"network-metrics-daemon-khgbr\" (UID: \"9f1c5761-f83c-4f1c-8994-8de155256444\") " pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:26.866473 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.866437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:26.869087 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.869061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6xv2\" (UniqueName: \"kubernetes.io/projected/89932788-5773-4bce-817e-a0041862d6a9-kube-api-access-k6xv2\") pod \"network-check-target-xfshv\" (UID: \"89932788-5773-4bce-817e-a0041862d6a9\") " pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:26.897806 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.897773    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:26.903941 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.903910    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-khgbr"
Sep 17 23:07:26.909094 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:26.909070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-csnkx"
Sep 17 23:07:27.218890 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:27.218826    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mllxt"
Sep 17 23:07:29.309295 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.309214    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fc9b5857-5v5zd" event={"ID":"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3","Type":"ContainerStarted","Data":"f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464"}
Sep 17 23:07:29.316347 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.316116    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x7bqz" event={"ID":"9bdaf271-bcf3-4c34-865f-4ec53f8d5c37","Type":"ContainerStarted","Data":"882c35dc28c03fadd4cef836011c54c1eeae43376b2cf23ef646b08e8a4a21b6"}
Sep 17 23:07:29.318945 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.318918    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tk7qr" event={"ID":"ac663cea-8971-4476-b0ef-fc259d52500f","Type":"ContainerStarted","Data":"1d2c9a20f6d0cdeefa95a6e10dbf64b3fddc1c78d85f6332af374c350abd92a9"}
Sep 17 23:07:29.320295 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.320274    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9sscn" event={"ID":"227b9f31-6478-4430-ae85-fa6085c296cd","Type":"ContainerStarted","Data":"3185aa26fa289d22d5fe7ddb5848374cf4e7cc5e59454005b9913dc67b3b0ae2"}
Sep 17 23:07:29.328280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.328236    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56fc9b5857-5v5zd" podStartSLOduration=1.557638491 podStartE2EDuration="7.328222989s" totalImagesPullingTime="5.770584498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:29.326366749 +0000 UTC m=+69.330558587" watchObservedRunningTime="2026-09-17 23:07:29.328222989 +0000 UTC m=+69.332414828"
Sep 17 23:07:29.344803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.344552    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x7bqz" podStartSLOduration=1.6393100440000001 podStartE2EDuration="7.344538401s" totalImagesPullingTime="5.705228357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:29.343835843 +0000 UTC m=+69.348027685" watchObservedRunningTime="2026-09-17 23:07:29.344538401 +0000 UTC m=+69.348730234"
Sep 17 23:07:29.376314 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.376286    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xfshv"]
Sep 17 23:07:29.380812 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.380709    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-csnkx"]
Sep 17 23:07:29.380812 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:29.380774    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-khgbr"]
Sep 17 23:07:29.381553 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:29.381524    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89932788_5773_4bce_817e_a0041862d6a9.slice/crio-50d746cf5056ed39c79e358c02225dc519774201012b214c3d011e858eb53eb5 WatchSource:0}: Error finding container 50d746cf5056ed39c79e358c02225dc519774201012b214c3d011e858eb53eb5: Status 404 returned error can't find the container with id 50d746cf5056ed39c79e358c02225dc519774201012b214c3d011e858eb53eb5
Sep 17 23:07:30.327611 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:30.327514    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khgbr" event={"ID":"9f1c5761-f83c-4f1c-8994-8de155256444","Type":"ContainerStarted","Data":"ac847d4e2f529846c1f5382a6cce3f532a4beb32b826b0d8069f1ff5e0b8b415"}
Sep 17 23:07:30.329109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:30.329028    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-csnkx" event={"ID":"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8","Type":"ContainerStarted","Data":"bcbab61a52819ce25b9d3e8ee8a99153e9a14f140105e67f1fbe6157790ec471"}
Sep 17 23:07:30.332060 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:30.332031    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9sscn" event={"ID":"227b9f31-6478-4430-ae85-fa6085c296cd","Type":"ContainerStarted","Data":"cb059bfe923aff1dce86d1ed6e3a71f9850f1057c9f9cdea264cc0236e837084"}
Sep 17 23:07:30.332744 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:30.332713    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:30.334490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:30.334466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xfshv" event={"ID":"89932788-5773-4bce-817e-a0041862d6a9","Type":"ContainerStarted","Data":"50d746cf5056ed39c79e358c02225dc519774201012b214c3d011e858eb53eb5"}
Sep 17 23:07:30.352020 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:30.351912    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9sscn" podStartSLOduration=3.596856113 podStartE2EDuration="8.351891874s" totalImagesPullingTime="4.755035761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:30.350573692 +0000 UTC m=+70.354765532" watchObservedRunningTime="2026-09-17 23:07:30.351891874 +0000 UTC m=+70.356083715"
Sep 17 23:07:31.342561 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:31.342443    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tk7qr" event={"ID":"ac663cea-8971-4476-b0ef-fc259d52500f","Type":"ContainerStarted","Data":"2ede62fc77b96d53036feb6045b51ba8b97e3872b414715a2765ad64763509e2"}
Sep 17 23:07:31.345411 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:31.345383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khgbr" event={"ID":"9f1c5761-f83c-4f1c-8994-8de155256444","Type":"ContainerStarted","Data":"49b363fa4d8999715c58d4a29449b30607c767d38bf4b915bdd517be59f1af98"}
Sep 17 23:07:31.345538 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:31.345432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-khgbr" event={"ID":"9f1c5761-f83c-4f1c-8994-8de155256444","Type":"ContainerStarted","Data":"0e25637ec15567b5f9f7f0d8ef1f9e548a102e60032d2472e1e167432f3f341f"}
Sep 17 23:07:31.366872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:31.366819    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tk7qr" podStartSLOduration=2.981426393 podStartE2EDuration="9.366798902s" totalImagesPullingTime="6.385372509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:07:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:31.3650574 +0000 UTC m=+71.369249240" watchObservedRunningTime="2026-09-17 23:07:31.366798902 +0000 UTC m=+71.370990743"
Sep 17 23:07:31.389765 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:31.389710    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-khgbr" podStartSLOduration=67.794894607 podStartE2EDuration="1m9.389694637s" totalImagesPullingTime="1.59480003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:31.38919072 +0000 UTC m=+71.393382562" watchObservedRunningTime="2026-09-17 23:07:31.389694637 +0000 UTC m=+71.393886476"
Sep 17 23:07:33.154191 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:33.154151    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:33.154632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:33.154228    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:33.159682 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:33.159657    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:33.357911 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:33.357882    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:07:34.357843 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:34.357750    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-csnkx" event={"ID":"50b8ad4b-7ba8-4b82-9a42-6acdf46910b8","Type":"ContainerStarted","Data":"3eafd399b485135a4202eb6ab9b2a3f5efc34168e6c70a9ecb380c2eec376541"}
Sep 17 23:07:34.359262 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:34.359231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xfshv" event={"ID":"89932788-5773-4bce-817e-a0041862d6a9","Type":"ContainerStarted","Data":"bb9220b2ede4a7b440aee6137544d1d57df6d4f1078d188fa37f371e78702478"}
Sep 17 23:07:34.378734 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:34.378694    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-csnkx" podStartSLOduration=67.695535441 podStartE2EDuration="1m12.378680303s" totalImagesPullingTime="4.683144862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:34.377877399 +0000 UTC m=+74.382069240" watchObservedRunningTime="2026-09-17 23:07:34.378680303 +0000 UTC m=+74.382872141"
Sep 17 23:07:34.397872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:34.397822    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xfshv" podStartSLOduration=67.714832879 podStartE2EDuration="1m12.397794692s" totalImagesPullingTime="4.682961813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:34.396781073 +0000 UTC m=+74.400972913" watchObservedRunningTime="2026-09-17 23:07:34.397794692 +0000 UTC m=+74.401986532"
Sep 17 23:07:35.366011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:35.365299    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:07:36.705836 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.705800    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7ttns"]
Sep 17 23:07:36.709957 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.709937    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.712961 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.712938    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 23:07:36.713114 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.713095    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5vsd4\""
Sep 17 23:07:36.713207 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.713170    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 23:07:36.713268 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.713033    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 23:07:36.713268 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.712982    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 23:07:36.714040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.714023    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 23:07:36.714161 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.714067    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 23:07:36.842442 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-sys\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.842616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-root\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.842616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842475    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b313c49a-3247-4725-ba81-5e936726b059-metrics-client-ca\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.842616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzzcr\" (UniqueName: \"kubernetes.io/projected/b313c49a-3247-4725-ba81-5e936726b059-kube-api-access-vzzcr\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.842616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.842616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842531    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-textfile\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.842616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842598    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.842796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-wtmp\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.842796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.842659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-accelerators-collector-config\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.943743 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.943918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-wtmp\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.943918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-accelerators-collector-config\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.943918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-sys\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.943918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943847    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-root\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.943918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b313c49a-3247-4725-ba81-5e936726b059-metrics-client-ca\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.943918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943911    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzzcr\" (UniqueName: \"kubernetes.io/projected/b313c49a-3247-4725-ba81-5e936726b059-kube-api-access-vzzcr\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.944238 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.944238 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-sys\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.944238 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943906    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-wtmp\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.944238 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943967    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-textfile\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.944238 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.943962    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b313c49a-3247-4725-ba81-5e936726b059-root\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.944238 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:36.944065    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 23:07:36.944238 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:36.944153    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls podName:b313c49a-3247-4725-ba81-5e936726b059 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:37.444112102 +0000 UTC m=+77.448303920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls") pod "node-exporter-7ttns" (UID: "b313c49a-3247-4725-ba81-5e936726b059") : secret "node-exporter-tls" not found
Sep 17 23:07:36.944508 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.944312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-textfile\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.944508 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.944455    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-accelerators-collector-config\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.944508 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.944502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b313c49a-3247-4725-ba81-5e936726b059-metrics-client-ca\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.946061 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.946036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:36.952017 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:36.951993    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzzcr\" (UniqueName: \"kubernetes.io/projected/b313c49a-3247-4725-ba81-5e936726b059-kube-api-access-vzzcr\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:37.447496 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.447461    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:37.447658 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:37.447571    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 23:07:37.447658 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:37.447656    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls podName:b313c49a-3247-4725-ba81-5e936726b059 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:38.447642558 +0000 UTC m=+78.451834375 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls") pod "node-exporter-7ttns" (UID: "b313c49a-3247-4725-ba81-5e936726b059") : secret "node-exporter-tls" not found
Sep 17 23:07:37.796857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.796779    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:07:37.801203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.801186    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.804015 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.803991    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 23:07:37.804220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.804076    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 23:07:37.804220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.804076    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 23:07:37.804220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.804107    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 23:07:37.804390 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.804357    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 23:07:37.804457 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.804406    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 23:07:37.804500 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.804410    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 23:07:37.804654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.804642    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 23:07:37.804891 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.804871    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dnwbp\""
Sep 17 23:07:37.805051 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.805038    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 23:07:37.820334 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.820314    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:07:37.951436 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951403    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-config-volume\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951436 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-config-out\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951455    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951664    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951688    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-web-config\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951714    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951772    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptslz\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-kube-api-access-ptslz\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951848    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:37.951922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:37.951871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053217 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053217 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053175    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053217 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053217 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-config-volume\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053478 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-config-out\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053478 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053478 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053311    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053478 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053478 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053687 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053795 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053768    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053929 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053826    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-web-config\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053929 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.053929 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.053918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptslz\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-kube-api-access-ptslz\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.055877 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:07:38.055615    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle podName:0438eed5-1c7a-4688-a363-ab48a0926d51 nodeName:}" failed. No retries permitted until 2026-09-17 23:07:38.555592492 +0000 UTC m=+78.559784329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51") : configmap references non-existent config key: ca-bundle.crt
Sep 17 23:07:38.055877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.055795    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.056240 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.056216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.056388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.056343    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-config-volume\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.056480 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.056419    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.056575 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.056525    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.056782 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.056763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.056893 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.056874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-config-out\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.057597 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.057578    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.057788 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.057771    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.058414 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.058395    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-web-config\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.064820 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.064801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptslz\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-kube-api-access-ptslz\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.457913 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.457875    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:38.460189 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.460170    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b313c49a-3247-4725-ba81-5e936726b059-node-exporter-tls\") pod \"node-exporter-7ttns\" (UID: \"b313c49a-3247-4725-ba81-5e936726b059\") " pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:38.518779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.518730    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7ttns"
Sep 17 23:07:38.530139 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:38.530100    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb313c49a_3247_4725_ba81_5e936726b059.slice/crio-6a947e9717e758513d3df95219d2943572866699d3e66e2cad6593d675d36cf2 WatchSource:0}: Error finding container 6a947e9717e758513d3df95219d2943572866699d3e66e2cad6593d675d36cf2: Status 404 returned error can't find the container with id 6a947e9717e758513d3df95219d2943572866699d3e66e2cad6593d675d36cf2
Sep 17 23:07:38.558313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.558287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.559015 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.558996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.709770 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.709692    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:07:38.994613 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:38.994590    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:07:38.995678 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:38.995655    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0438eed5_1c7a_4688_a363_ab48a0926d51.slice/crio-8a28e63e598f7f40ae077292fa886c09879bff99c9516dcf49447d91b22d030a WatchSource:0}: Error finding container 8a28e63e598f7f40ae077292fa886c09879bff99c9516dcf49447d91b22d030a: Status 404 returned error can't find the container with id 8a28e63e598f7f40ae077292fa886c09879bff99c9516dcf49447d91b22d030a
Sep 17 23:07:39.381346 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.381297    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7ttns" event={"ID":"b313c49a-3247-4725-ba81-5e936726b059","Type":"ContainerStarted","Data":"6a947e9717e758513d3df95219d2943572866699d3e66e2cad6593d675d36cf2"}
Sep 17 23:07:39.382736 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.382708    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerStarted","Data":"8a28e63e598f7f40ae077292fa886c09879bff99c9516dcf49447d91b22d030a"}
Sep 17 23:07:39.688424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.688394    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d899f7d48-96n75"]
Sep 17 23:07:39.692589 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.692571    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.695606 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.695421    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 17 23:07:39.695606 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.695442    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 17 23:07:39.695606 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.695477    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-aha2q2tt3nuo5\""
Sep 17 23:07:39.695606 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.695577    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-l9b8n\""
Sep 17 23:07:39.695606 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.695587    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 17 23:07:39.695907 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.695848    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 17 23:07:39.696377 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.696353    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 17 23:07:39.706459 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.706435    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d899f7d48-96n75"]
Sep 17 23:07:39.769050 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.769022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.769207 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.769077    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-grpc-tls\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.769207 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.769163    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnqkk\" (UniqueName: \"kubernetes.io/projected/abcd80ee-e836-4577-92f4-97722f9b41b1-kube-api-access-qnqkk\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.769312 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.769243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.769355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.769306    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.769355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.769333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-tls\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.769452 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.769364    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abcd80ee-e836-4577-92f4-97722f9b41b1-metrics-client-ca\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.769452 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.769391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.870110 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.870076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-grpc-tls\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.870292 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.870142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnqkk\" (UniqueName: \"kubernetes.io/projected/abcd80ee-e836-4577-92f4-97722f9b41b1-kube-api-access-qnqkk\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.870292 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.870169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.870292 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.870209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.870292 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.870241    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-tls\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.870292 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.870274    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abcd80ee-e836-4577-92f4-97722f9b41b1-metrics-client-ca\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.870534 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.870311    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.870534 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.870348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.871427 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.871389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abcd80ee-e836-4577-92f4-97722f9b41b1-metrics-client-ca\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.873285 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.873258    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.873660 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.873635    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-grpc-tls\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.873744 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.873677    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-tls\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.873959 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.873939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.874043 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.874020    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.874377 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.874352    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/abcd80ee-e836-4577-92f4-97722f9b41b1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:39.878935 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:39.878909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnqkk\" (UniqueName: \"kubernetes.io/projected/abcd80ee-e836-4577-92f4-97722f9b41b1-kube-api-access-qnqkk\") pod \"thanos-querier-7d899f7d48-96n75\" (UID: \"abcd80ee-e836-4577-92f4-97722f9b41b1\") " pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:40.041411 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:40.041317    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:40.387372 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:40.387333    2581 generic.go:356] "Generic (PLEG): container finished" podID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerID="b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126" exitCode=0
Sep 17 23:07:40.387543 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:40.387426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerDied","Data":"b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126"}
Sep 17 23:07:40.389112 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:40.389050    2581 generic.go:356] "Generic (PLEG): container finished" podID="b313c49a-3247-4725-ba81-5e936726b059" containerID="bf39eaba82324b9048afe2803cadb9d897017322526697c10543cce219e4897a" exitCode=0
Sep 17 23:07:40.389112 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:40.389106    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7ttns" event={"ID":"b313c49a-3247-4725-ba81-5e936726b059","Type":"ContainerDied","Data":"bf39eaba82324b9048afe2803cadb9d897017322526697c10543cce219e4897a"}
Sep 17 23:07:40.444671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:40.444516    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d899f7d48-96n75"]
Sep 17 23:07:40.446664 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:40.446645    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabcd80ee_e836_4577_92f4_97722f9b41b1.slice/crio-16600b4b981b96234818dd77888bdfb98ca96dfa103801141934ee86d0b7aa45 WatchSource:0}: Error finding container 16600b4b981b96234818dd77888bdfb98ca96dfa103801141934ee86d0b7aa45: Status 404 returned error can't find the container with id 16600b4b981b96234818dd77888bdfb98ca96dfa103801141934ee86d0b7aa45
Sep 17 23:07:41.350346 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.350314    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9sscn"
Sep 17 23:07:41.396633 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.396593    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7ttns" event={"ID":"b313c49a-3247-4725-ba81-5e936726b059","Type":"ContainerStarted","Data":"44c2ed83f30f8a760d8705c4421ba1fc45f6cf19ca6378e6f99361d33a257598"}
Sep 17 23:07:41.396778 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.396660    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7ttns" event={"ID":"b313c49a-3247-4725-ba81-5e936726b059","Type":"ContainerStarted","Data":"221edb387521c02f0a16d1c55e48440d496146cca2317ef84c0d7b3e0cef5248"}
Sep 17 23:07:41.399613 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.399584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75" event={"ID":"abcd80ee-e836-4577-92f4-97722f9b41b1","Type":"ContainerStarted","Data":"16600b4b981b96234818dd77888bdfb98ca96dfa103801141934ee86d0b7aa45"}
Sep 17 23:07:41.424611 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.424552    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7ttns" podStartSLOduration=4.353603253 podStartE2EDuration="5.424531826s" totalImagesPullingTime="1.008043177s" totalInitContainerRuntime="62.885396ms" isStatefulPod=true podCreationTimestamp="2026-09-17 23:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:41.423562977 +0000 UTC m=+81.427754809" watchObservedRunningTime="2026-09-17 23:07:41.424531826 +0000 UTC m=+81.428723666"
Sep 17 23:07:41.956811 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.956778    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-545f84c569-5cz58"]
Sep 17 23:07:41.960295 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.960273    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:41.963259 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.963229    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w6qdj\""
Sep 17 23:07:41.963516 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.963497    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 17 23:07:41.963622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.963510    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 17 23:07:41.963622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.963532    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 17 23:07:41.963622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.963546    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 17 23:07:41.963622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.963555    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 17 23:07:41.969492 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.969469    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 17 23:07:41.977995 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:41.977952    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-545f84c569-5cz58"]
Sep 17 23:07:42.091339 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.091305    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-federate-client-tls\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.091518 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.091373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-serving-certs-ca-bundle\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.091518 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.091398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbpcc\" (UniqueName: \"kubernetes.io/projected/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-kube-api-access-hbpcc\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.091518 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.091487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.091681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.091522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-metrics-client-ca\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.091681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.091552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.091681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.091571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-telemeter-client-tls\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.091681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.091594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-secret-telemeter-client\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192089 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-serving-certs-ca-bundle\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192275 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbpcc\" (UniqueName: \"kubernetes.io/projected/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-kube-api-access-hbpcc\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192275 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192275 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-metrics-client-ca\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192275 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192275 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-telemeter-client-tls\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192478 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-secret-telemeter-client\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192478 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192317    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-federate-client-tls\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.192789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.192760    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-serving-certs-ca-bundle\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.193311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.193281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-metrics-client-ca\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.193428 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.193342    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.195364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.195342    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-telemeter-client-tls\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.195462 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.195381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.195524 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.195491    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-secret-telemeter-client\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.195568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.195519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-federate-client-tls\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.200638 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.200616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbpcc\" (UniqueName: \"kubernetes.io/projected/c00e524b-5b91-40e0-ba70-5a59b1d9f96d-kube-api-access-hbpcc\") pod \"telemeter-client-545f84c569-5cz58\" (UID: \"c00e524b-5b91-40e0-ba70-5a59b1d9f96d\") " pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.271571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.271491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-545f84c569-5cz58"
Sep 17 23:07:42.710102 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:42.710072    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-545f84c569-5cz58"]
Sep 17 23:07:42.712174 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:42.712148    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00e524b_5b91_40e0_ba70_5a59b1d9f96d.slice/crio-f25198130db5b1bdc2ac1f1047c6afc8038e92593b70f59766608581e477f412 WatchSource:0}: Error finding container f25198130db5b1bdc2ac1f1047c6afc8038e92593b70f59766608581e477f412: Status 404 returned error can't find the container with id f25198130db5b1bdc2ac1f1047c6afc8038e92593b70f59766608581e477f412
Sep 17 23:07:43.412723 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.412464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerStarted","Data":"05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7"}
Sep 17 23:07:43.412723 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.412554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerStarted","Data":"f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973"}
Sep 17 23:07:43.412723 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.412596    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerStarted","Data":"2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0"}
Sep 17 23:07:43.412723 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.412637    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerStarted","Data":"34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c"}
Sep 17 23:07:43.412723 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.412675    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerStarted","Data":"27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505"}
Sep 17 23:07:43.414135 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.414056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-545f84c569-5cz58" event={"ID":"c00e524b-5b91-40e0-ba70-5a59b1d9f96d","Type":"ContainerStarted","Data":"f25198130db5b1bdc2ac1f1047c6afc8038e92593b70f59766608581e477f412"}
Sep 17 23:07:43.416986 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.416823    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75" event={"ID":"abcd80ee-e836-4577-92f4-97722f9b41b1","Type":"ContainerStarted","Data":"37e9419407598a586a6282fd06f0b95baa01ebdc6aec04ef7e3068dde1e3cdcd"}
Sep 17 23:07:43.416986 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.416898    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75" event={"ID":"abcd80ee-e836-4577-92f4-97722f9b41b1","Type":"ContainerStarted","Data":"b42e334b37902b84d6cddf996161e85078018a954da569c8bdf1aa92b957a6f9"}
Sep 17 23:07:43.416986 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:43.416940    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75" event={"ID":"abcd80ee-e836-4577-92f4-97722f9b41b1","Type":"ContainerStarted","Data":"64860c3ed2f35b3de8258c999c32a91ea7e97547e2a53e290d21625abce2939f"}
Sep 17 23:07:44.425610 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:44.425572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75" event={"ID":"abcd80ee-e836-4577-92f4-97722f9b41b1","Type":"ContainerStarted","Data":"b9e9b8579168562d8d398eddccb9921517446ffe532d386bd68be0b94c44c0d3"}
Sep 17 23:07:44.426113 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:44.425654    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75" event={"ID":"abcd80ee-e836-4577-92f4-97722f9b41b1","Type":"ContainerStarted","Data":"d898caa0832c92cc2551c1f27014545615fc8a5474239f70d924ce4585f6a55c"}
Sep 17 23:07:44.426113 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:44.425698    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:44.426113 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:44.425736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75" event={"ID":"abcd80ee-e836-4577-92f4-97722f9b41b1","Type":"ContainerStarted","Data":"5a9c6226c1c6f7e70979ca7a01cf1f1dc18cd375d1de8af27b112f25682b6e20"}
Sep 17 23:07:44.429679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:44.429653    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerStarted","Data":"d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8"}
Sep 17 23:07:44.452778 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:44.452730    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75" podStartSLOduration=2.582695377 podStartE2EDuration="5.452710671s" totalImagesPullingTime="2.870015294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:44.451160589 +0000 UTC m=+84.455352428" watchObservedRunningTime="2026-09-17 23:07:44.452710671 +0000 UTC m=+84.456902513"
Sep 17 23:07:44.481794 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:44.481742    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.433588505 podStartE2EDuration="7.481724856s" totalImagesPullingTime="4.031400473s" totalInitContainerRuntime="16.735878ms" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:44.480470821 +0000 UTC m=+84.484662662" watchObservedRunningTime="2026-09-17 23:07:44.481724856 +0000 UTC m=+84.485916695"
Sep 17 23:07:45.434997 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.434955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-545f84c569-5cz58" event={"ID":"c00e524b-5b91-40e0-ba70-5a59b1d9f96d","Type":"ContainerStarted","Data":"0a8c06522b80dba2f7fe5734e9249c1306dc180e7e5768758c1f8aa6215e65d3"}
Sep 17 23:07:45.435392 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.435042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-545f84c569-5cz58" event={"ID":"c00e524b-5b91-40e0-ba70-5a59b1d9f96d","Type":"ContainerStarted","Data":"b5432bd45089e8040fd084f92d6842cbbbb34945837e5a46491019b1dda934de"}
Sep 17 23:07:45.435392 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.435061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-545f84c569-5cz58" event={"ID":"c00e524b-5b91-40e0-ba70-5a59b1d9f96d","Type":"ContainerStarted","Data":"26159639b5dd5dd09c2860dd38d1a3b486d67852d571afc2f44195c5ed93c980"}
Sep 17 23:07:45.459529 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.459481    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-545f84c569-5cz58" podStartSLOduration=2.635959139 podStartE2EDuration="4.459464947s" totalImagesPullingTime="1.823505808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:45.458304225 +0000 UTC m=+85.462496063" watchObservedRunningTime="2026-09-17 23:07:45.459464947 +0000 UTC m=+85.463656783"
Sep 17 23:07:45.615142 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.615089    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64f564744c-dst2h"]
Sep 17 23:07:45.618302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.618280    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.625572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.625547    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 23:07:45.627336 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.627313    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f564744c-dst2h"]
Sep 17 23:07:45.725683 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.725606    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzjgp\" (UniqueName: \"kubernetes.io/projected/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-kube-api-access-bzjgp\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.725683 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.725640    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-serving-cert\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.725683 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.725663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-service-ca\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.725954 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.725759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-oauth-serving-cert\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.725954 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.725814    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-trusted-ca-bundle\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.725954 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.725856    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-config\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.725954 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.725884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-oauth-config\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.826589 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.826562    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-trusted-ca-bundle\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.826705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.826601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-config\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.826705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.826621    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-oauth-config\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.826705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.826663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzjgp\" (UniqueName: \"kubernetes.io/projected/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-kube-api-access-bzjgp\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.826705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.826683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-serving-cert\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.826705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.826699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-service-ca\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.826898 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.826723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-oauth-serving-cert\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.827389 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.827356    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-config\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.827517 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.827466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-service-ca\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.827575 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.827554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-trusted-ca-bundle\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.827612 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.827567    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-oauth-serving-cert\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.829207 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.829184    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-serving-cert\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.829301 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.829280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-oauth-config\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.835513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.835489    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzjgp\" (UniqueName: \"kubernetes.io/projected/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-kube-api-access-bzjgp\") pod \"console-64f564744c-dst2h\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") " pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:45.927650 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:45.927608    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:46.202186 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:46.202153    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f564744c-dst2h"]
Sep 17 23:07:46.204961 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:07:46.204935    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a32d2fd_5b87_4b22_b61d_b9e8f7480b4a.slice/crio-163c750f7d0403c6edba98c37e137c621baea8c1dc9f5552dc9c8b9fae98963c WatchSource:0}: Error finding container 163c750f7d0403c6edba98c37e137c621baea8c1dc9f5552dc9c8b9fae98963c: Status 404 returned error can't find the container with id 163c750f7d0403c6edba98c37e137c621baea8c1dc9f5552dc9c8b9fae98963c
Sep 17 23:07:46.297814 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:46.297788    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7646669d8b-fwlsv"
Sep 17 23:07:46.440312 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:46.440276    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f564744c-dst2h" event={"ID":"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a","Type":"ContainerStarted","Data":"002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b"}
Sep 17 23:07:46.440667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:46.440327    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f564744c-dst2h" event={"ID":"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a","Type":"ContainerStarted","Data":"163c750f7d0403c6edba98c37e137c621baea8c1dc9f5552dc9c8b9fae98963c"}
Sep 17 23:07:46.463391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:46.463341    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64f564744c-dst2h" podStartSLOduration=1.463326066 podStartE2EDuration="1.463326066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:07:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:07:46.462172408 +0000 UTC m=+86.466364244" watchObservedRunningTime="2026-09-17 23:07:46.463326066 +0000 UTC m=+86.467517905"
Sep 17 23:07:50.441393 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:50.441363    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d899f7d48-96n75"
Sep 17 23:07:55.928143 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:55.928084    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:55.928508 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:55.928161    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:55.932824 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:55.932800    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:56.482066 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:56.482033    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:07:56.532144 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:07:56.532084    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56fc9b5857-5v5zd"]
Sep 17 23:08:06.370449 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:06.370420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xfshv"
Sep 17 23:08:11.202111 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:11.202083    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x7bqz_9bdaf271-bcf3-4c34-865f-4ec53f8d5c37/serve-healthcheck-canary/0.log"
Sep 17 23:08:21.556093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.556025    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56fc9b5857-5v5zd" podUID="7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" containerName="console" containerID="cri-o://f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464" gracePeriod=15
Sep 17 23:08:21.795741 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.795719    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:08:21.892844 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.892813    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-oauth-serving-cert\") pod \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") "
Sep 17 23:08:21.893045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.892853    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9p9x\" (UniqueName: \"kubernetes.io/projected/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-kube-api-access-t9p9x\") pod \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") "
Sep 17 23:08:21.893045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.892878    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-config\") pod \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") "
Sep 17 23:08:21.893045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.892942    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-service-ca\") pod \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") "
Sep 17 23:08:21.893045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.892976    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-oauth-config\") pod \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") "
Sep 17 23:08:21.893045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.892995    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-serving-cert\") pod \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\" (UID: \"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3\") "
Sep 17 23:08:21.893326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.893235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" (UID: "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:08:21.893326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.893270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-config" (OuterVolumeSpecName: "console-config") pod "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" (UID: "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:08:21.893326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.893291    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-service-ca" (OuterVolumeSpecName: "service-ca") pod "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" (UID: "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:08:21.895449 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.895423    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" (UID: "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:21.895574 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.895462    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-kube-api-access-t9p9x" (OuterVolumeSpecName: "kube-api-access-t9p9x") pod "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" (UID: "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3"). InnerVolumeSpecName "kube-api-access-t9p9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:08:21.895574 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.895550    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" (UID: "7f4233bc-3012-4a93-9e5d-a0a07a57b8e3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:21.994531 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.994503    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:21.994531 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.994533    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-oauth-config\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:21.994728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.994549    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-serving-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:21.994728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.994564    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-oauth-serving-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:21.994728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.994579    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9p9x\" (UniqueName: \"kubernetes.io/projected/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-kube-api-access-t9p9x\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:21.994728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:21.994593    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3-console-config\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:22.575223 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.575190    2581 generic.go:356] "Generic (PLEG): container finished" podID="7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" containerID="f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464" exitCode=0
Sep 17 23:08:22.575607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.575244    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fc9b5857-5v5zd"
Sep 17 23:08:22.575607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.575233    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fc9b5857-5v5zd" event={"ID":"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3","Type":"ContainerDied","Data":"f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464"}
Sep 17 23:08:22.575607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.575326    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fc9b5857-5v5zd" event={"ID":"7f4233bc-3012-4a93-9e5d-a0a07a57b8e3","Type":"ContainerDied","Data":"72226ada883d8fd68b3a4bd29d84f0c034d366fe49fde956e16d6c2c6054edea"}
Sep 17 23:08:22.575607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.575372    2581 scope.go:117] "RemoveContainer" containerID="f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464"
Sep 17 23:08:22.587519 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.587471    2581 scope.go:117] "RemoveContainer" containerID="f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464"
Sep 17 23:08:22.587896 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:08:22.587869    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464\": container with ID starting with f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464 not found: ID does not exist" containerID="f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464"
Sep 17 23:08:22.588010 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.587904    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464"} err="failed to get container status \"f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464\": rpc error: code = NotFound desc = could not find container \"f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464\": container with ID starting with f0fa88ed7de3c8b2c8809158d6d86a2a8edf3684aa74abae8f5bf96dbd230464 not found: ID does not exist"
Sep 17 23:08:22.596963 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.596932    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56fc9b5857-5v5zd"]
Sep 17 23:08:22.603878 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:22.603851    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56fc9b5857-5v5zd"]
Sep 17 23:08:23.992443 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:23.992410    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" path="/var/lib/kubelet/pods/7f4233bc-3012-4a93-9e5d-a0a07a57b8e3/volumes"
Sep 17 23:08:57.080631 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.080596    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:08:57.081150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.080905    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="alertmanager" containerID="cri-o://27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505" gracePeriod=120
Sep 17 23:08:57.081150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.080971    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy-web" containerID="cri-o://2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0" gracePeriod=120
Sep 17 23:08:57.081150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.081045    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="config-reloader" containerID="cri-o://34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c" gracePeriod=120
Sep 17 23:08:57.081150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.081058    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy" containerID="cri-o://f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973" gracePeriod=120
Sep 17 23:08:57.081150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.081098    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="prom-label-proxy" containerID="cri-o://d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8" gracePeriod=120
Sep 17 23:08:57.081414 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.080963    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy-metric" containerID="cri-o://05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7" gracePeriod=120
Sep 17 23:08:57.702484 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702452    2581 generic.go:356] "Generic (PLEG): container finished" podID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerID="d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8" exitCode=0
Sep 17 23:08:57.702484 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702474    2581 generic.go:356] "Generic (PLEG): container finished" podID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerID="05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7" exitCode=0
Sep 17 23:08:57.702484 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702480    2581 generic.go:356] "Generic (PLEG): container finished" podID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerID="f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973" exitCode=0
Sep 17 23:08:57.702484 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702486    2581 generic.go:356] "Generic (PLEG): container finished" podID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerID="34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c" exitCode=0
Sep 17 23:08:57.702484 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702493    2581 generic.go:356] "Generic (PLEG): container finished" podID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerID="27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505" exitCode=0
Sep 17 23:08:57.702768 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702522    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerDied","Data":"d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8"}
Sep 17 23:08:57.702768 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702592    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerDied","Data":"05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7"}
Sep 17 23:08:57.702768 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702619    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerDied","Data":"f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973"}
Sep 17 23:08:57.702768 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702644    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerDied","Data":"34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c"}
Sep 17 23:08:57.702768 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:57.702674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerDied","Data":"27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505"}
Sep 17 23:08:58.324584 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.324562    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.480299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480200    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-main-db\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480246    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-config-volume\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480266    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-web\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480291    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-config-out\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480312    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-metrics-client-ca\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480341    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480382    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-web-config\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480413    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-main-tls\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480436    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptslz\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-kube-api-access-ptslz\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480466    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-tls-assets\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480506    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480541    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480564    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-cluster-tls-config\") pod \"0438eed5-1c7a-4688-a363-ab48a0926d51\" (UID: \"0438eed5-1c7a-4688-a363-ab48a0926d51\") "
Sep 17 23:08:58.480671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480620    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:08:58.481109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480715    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:08:58.481109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480826    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-main-db\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.481109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.480843    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-metrics-client-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.481109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.481075    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:08:58.483099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.483056    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-config-volume" (OuterVolumeSpecName: "config-volume") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:58.483562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.483525    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:08:58.483670 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.483585    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:58.483670 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.483644    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:58.483808 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.483781    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-config-out" (OuterVolumeSpecName: "config-out") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:08:58.484051 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.484031    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-kube-api-access-ptslz" (OuterVolumeSpecName: "kube-api-access-ptslz") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "kube-api-access-ptslz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:08:58.484255 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.484224    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:58.484483 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.484456    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:58.487670 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.487640    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:58.493459 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.493437    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-web-config" (OuterVolumeSpecName: "web-config") pod "0438eed5-1c7a-4688-a363-ab48a0926d51" (UID: "0438eed5-1c7a-4688-a363-ab48a0926d51"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:08:58.581956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.581915    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.581956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.581954    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.581956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.581965    2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-cluster-tls-config\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.582172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.581974    2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.582172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.581982    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.582172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.581990    2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0438eed5-1c7a-4688-a363-ab48a0926d51-config-out\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.582172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.581998    2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0438eed5-1c7a-4688-a363-ab48a0926d51-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.582172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.582013    2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-web-config\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.582172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.582021    2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0438eed5-1c7a-4688-a363-ab48a0926d51-secret-alertmanager-main-tls\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.582172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.582029    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptslz\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-kube-api-access-ptslz\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.582172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.582037    2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0438eed5-1c7a-4688-a363-ab48a0926d51-tls-assets\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:08:58.711489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.711457    2581 generic.go:356] "Generic (PLEG): container finished" podID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerID="2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0" exitCode=0
Sep 17 23:08:58.711651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.711509    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerDied","Data":"2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0"}
Sep 17 23:08:58.711651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.711544    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.711651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.711572    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0438eed5-1c7a-4688-a363-ab48a0926d51","Type":"ContainerDied","Data":"8a28e63e598f7f40ae077292fa886c09879bff99c9516dcf49447d91b22d030a"}
Sep 17 23:08:58.711651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.711612    2581 scope.go:117] "RemoveContainer" containerID="d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8"
Sep 17 23:08:58.719040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.719022    2581 scope.go:117] "RemoveContainer" containerID="05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7"
Sep 17 23:08:58.725843 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.725827    2581 scope.go:117] "RemoveContainer" containerID="f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973"
Sep 17 23:08:58.732259 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.732241    2581 scope.go:117] "RemoveContainer" containerID="2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0"
Sep 17 23:08:58.736153 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.736118    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:08:58.739469 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.739451    2581 scope.go:117] "RemoveContainer" containerID="34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c"
Sep 17 23:08:58.740306 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.740287    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:08:58.745835 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.745818    2581 scope.go:117] "RemoveContainer" containerID="27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505"
Sep 17 23:08:58.752060 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.752044    2581 scope.go:117] "RemoveContainer" containerID="b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126"
Sep 17 23:08:58.758741 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.758726    2581 scope.go:117] "RemoveContainer" containerID="d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8"
Sep 17 23:08:58.758997 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:08:58.758977    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8\": container with ID starting with d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8 not found: ID does not exist" containerID="d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8"
Sep 17 23:08:58.759059 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.759008    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8"} err="failed to get container status \"d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8\": rpc error: code = NotFound desc = could not find container \"d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8\": container with ID starting with d2a3b5c5e273569559686ee330cd079ecb1f9ce5667cbe7fd9d67674c0c11ca8 not found: ID does not exist"
Sep 17 23:08:58.759059 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.759033    2581 scope.go:117] "RemoveContainer" containerID="05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7"
Sep 17 23:08:58.759305 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:08:58.759287    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7\": container with ID starting with 05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7 not found: ID does not exist" containerID="05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7"
Sep 17 23:08:58.759343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.759311    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7"} err="failed to get container status \"05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7\": rpc error: code = NotFound desc = could not find container \"05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7\": container with ID starting with 05093fea8814f84a41b5d14558a7609350c910993dc35481c836b6b1dc7627e7 not found: ID does not exist"
Sep 17 23:08:58.759343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.759332    2581 scope.go:117] "RemoveContainer" containerID="f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973"
Sep 17 23:08:58.759567 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:08:58.759550    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973\": container with ID starting with f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973 not found: ID does not exist" containerID="f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973"
Sep 17 23:08:58.759608 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.759581    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973"} err="failed to get container status \"f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973\": rpc error: code = NotFound desc = could not find container \"f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973\": container with ID starting with f8633f54ce6e36ce97251e523f135df634e2e8f85016b51bf3efa0f89f97d973 not found: ID does not exist"
Sep 17 23:08:58.759608 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.759596    2581 scope.go:117] "RemoveContainer" containerID="2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0"
Sep 17 23:08:58.759840 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:08:58.759823    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0\": container with ID starting with 2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0 not found: ID does not exist" containerID="2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0"
Sep 17 23:08:58.759898 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.759846    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0"} err="failed to get container status \"2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0\": rpc error: code = NotFound desc = could not find container \"2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0\": container with ID starting with 2f1ce7bfd23e2428ec92d299ccd5ea3da95a282043f8e63343f09b1d8dc5a5b0 not found: ID does not exist"
Sep 17 23:08:58.759898 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.759865    2581 scope.go:117] "RemoveContainer" containerID="34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c"
Sep 17 23:08:58.760151 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:08:58.760116    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c\": container with ID starting with 34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c not found: ID does not exist" containerID="34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c"
Sep 17 23:08:58.760196 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.760155    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c"} err="failed to get container status \"34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c\": rpc error: code = NotFound desc = could not find container \"34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c\": container with ID starting with 34e285c82ea66bb6db147015a4a1c8db8f7dddfb7dabf5fb287f4eb4124eaa7c not found: ID does not exist"
Sep 17 23:08:58.760196 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.760168    2581 scope.go:117] "RemoveContainer" containerID="27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505"
Sep 17 23:08:58.760413 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:08:58.760398    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505\": container with ID starting with 27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505 not found: ID does not exist" containerID="27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505"
Sep 17 23:08:58.760451 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.760417    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505"} err="failed to get container status \"27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505\": rpc error: code = NotFound desc = could not find container \"27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505\": container with ID starting with 27cd8819caf4215a1e829335bb831edcdc5fb4bc22498c49429e40f5d888f505 not found: ID does not exist"
Sep 17 23:08:58.760451 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.760433    2581 scope.go:117] "RemoveContainer" containerID="b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126"
Sep 17 23:08:58.760675 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:08:58.760659    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126\": container with ID starting with b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126 not found: ID does not exist" containerID="b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126"
Sep 17 23:08:58.760722 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.760677    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126"} err="failed to get container status \"b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126\": rpc error: code = NotFound desc = could not find container \"b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126\": container with ID starting with b863b4be3953e69cb1f795d3d0d7b14aad294cbc2d92a92435a5a1f15c89c126 not found: ID does not exist"
Sep 17 23:08:58.765979 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.765959    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:08:58.766218 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766207    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy-metric"
Sep 17 23:08:58.766264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766220    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy-metric"
Sep 17 23:08:58.766264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766229    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" containerName="console"
Sep 17 23:08:58.766264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766234    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" containerName="console"
Sep 17 23:08:58.766264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766241    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="prom-label-proxy"
Sep 17 23:08:58.766264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766247    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="prom-label-proxy"
Sep 17 23:08:58.766264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766254    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="init-config-reloader"
Sep 17 23:08:58.766264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766259    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="init-config-reloader"
Sep 17 23:08:58.766264 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766265    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy-web"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766270    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy-web"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766276    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="alertmanager"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766281    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="alertmanager"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766294    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766299    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766306    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="config-reloader"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766311    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="config-reloader"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766347    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f4233bc-3012-4a93-9e5d-a0a07a57b8e3" containerName="console"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766353    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy-web"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766360    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="prom-label-proxy"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766366    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="config-reloader"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766373    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766378    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="alertmanager"
Sep 17 23:08:58.766490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.766383    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" containerName="kube-rbac-proxy-metric"
Sep 17 23:08:58.771093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.771079    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.773555 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.773535    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 17 23:08:58.773655 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.773556    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dnwbp\""
Sep 17 23:08:58.773655 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.773563    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 17 23:08:58.773655 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.773570    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 17 23:08:58.773655 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.773625    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 17 23:08:58.773872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.773687    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 17 23:08:58.773872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.773773    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 17 23:08:58.774026 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.774012    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 17 23:08:58.774073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.774035    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 17 23:08:58.780116 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.780080    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 17 23:08:58.787110 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.787092    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:08:58.884171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884118    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9ff4ec7-81a4-4e14-87b0-01f028df392b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884201    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884221    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9ff4ec7-81a4-4e14-87b0-01f028df392b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9ff4ec7-81a4-4e14-87b0-01f028df392b-config-out\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884265    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884283    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884298    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5fr4\" (UniqueName: \"kubernetes.io/projected/e9ff4ec7-81a4-4e14-87b0-01f028df392b-kube-api-access-s5fr4\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-web-config\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9ff4ec7-81a4-4e14-87b0-01f028df392b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884446    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9ff4ec7-81a4-4e14-87b0-01f028df392b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.884607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.884464    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985523    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985563    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9ff4ec7-81a4-4e14-87b0-01f028df392b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985587    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9ff4ec7-81a4-4e14-87b0-01f028df392b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9ff4ec7-81a4-4e14-87b0-01f028df392b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9ff4ec7-81a4-4e14-87b0-01f028df392b-config-out\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985815    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5fr4\" (UniqueName: \"kubernetes.io/projected/e9ff4ec7-81a4-4e14-87b0-01f028df392b-kube-api-access-s5fr4\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-web-config\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.985919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.985889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9ff4ec7-81a4-4e14-87b0-01f028df392b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.986370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.986198    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9ff4ec7-81a4-4e14-87b0-01f028df392b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.986904 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.986485    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9ff4ec7-81a4-4e14-87b0-01f028df392b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.987414 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.987385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9ff4ec7-81a4-4e14-87b0-01f028df392b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.988692 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.988665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9ff4ec7-81a4-4e14-87b0-01f028df392b-config-out\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.988848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.988795    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9ff4ec7-81a4-4e14-87b0-01f028df392b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.988981 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.988956    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.989112 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.989091    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.989199 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.989179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.989370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.989349    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-web-config\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.989672 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.989650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.989946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.989930    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.990428 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.990407    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9ff4ec7-81a4-4e14-87b0-01f028df392b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:58.993874 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:58.993854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5fr4\" (UniqueName: \"kubernetes.io/projected/e9ff4ec7-81a4-4e14-87b0-01f028df392b-kube-api-access-s5fr4\") pod \"alertmanager-main-0\" (UID: \"e9ff4ec7-81a4-4e14-87b0-01f028df392b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:59.081100 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:59.081070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 17 23:08:59.360391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:59.360364    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 17 23:08:59.362163 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:08:59.362107    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9ff4ec7_81a4_4e14_87b0_01f028df392b.slice/crio-4c9f4a3df6f99e4a4f68489e7672db7c4b7af92ed90d71a2685c6cd299b55f0d WatchSource:0}: Error finding container 4c9f4a3df6f99e4a4f68489e7672db7c4b7af92ed90d71a2685c6cd299b55f0d: Status 404 returned error can't find the container with id 4c9f4a3df6f99e4a4f68489e7672db7c4b7af92ed90d71a2685c6cd299b55f0d
Sep 17 23:08:59.716897 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:59.716810    2581 generic.go:356] "Generic (PLEG): container finished" podID="e9ff4ec7-81a4-4e14-87b0-01f028df392b" containerID="95af58ff6786ac81322b0987813910e0db00b22b0ad6586f4883f37e49e1edc4" exitCode=0
Sep 17 23:08:59.717033 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:59.716897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9ff4ec7-81a4-4e14-87b0-01f028df392b","Type":"ContainerDied","Data":"95af58ff6786ac81322b0987813910e0db00b22b0ad6586f4883f37e49e1edc4"}
Sep 17 23:08:59.717033 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:59.716967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9ff4ec7-81a4-4e14-87b0-01f028df392b","Type":"ContainerStarted","Data":"4c9f4a3df6f99e4a4f68489e7672db7c4b7af92ed90d71a2685c6cd299b55f0d"}
Sep 17 23:08:59.992062 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:08:59.992036    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0438eed5-1c7a-4688-a363-ab48a0926d51" path="/var/lib/kubelet/pods/0438eed5-1c7a-4688-a363-ab48a0926d51/volumes"
Sep 17 23:09:00.725513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:00.725482    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9ff4ec7-81a4-4e14-87b0-01f028df392b","Type":"ContainerStarted","Data":"f8baded2848086129559fd731d1cdf17c8e9a04e91c9b6a5189c27547a361eb1"}
Sep 17 23:09:00.725877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:00.725545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9ff4ec7-81a4-4e14-87b0-01f028df392b","Type":"ContainerStarted","Data":"5606b0a5bd4065e60982953cfff9f1fbb2e15e3c83793b1dc2b935f5846e263d"}
Sep 17 23:09:00.725877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:00.725573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9ff4ec7-81a4-4e14-87b0-01f028df392b","Type":"ContainerStarted","Data":"033da0bccec532704fcc53e8f9c4069a8bb505e85278aeebf267e8fb2610e77d"}
Sep 17 23:09:00.725877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:00.725605    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9ff4ec7-81a4-4e14-87b0-01f028df392b","Type":"ContainerStarted","Data":"b2bb5224fa1493d3bbe9b286e0825aa6de2e7afbf3a4ed5cbd92b7b08f6c6df8"}
Sep 17 23:09:00.725877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:00.725628    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9ff4ec7-81a4-4e14-87b0-01f028df392b","Type":"ContainerStarted","Data":"1e147f9a30b16560eea71f916dcdea8f8701c0fbec277e15cd699331496b8e59"}
Sep 17 23:09:00.725877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:00.725652    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9ff4ec7-81a4-4e14-87b0-01f028df392b","Type":"ContainerStarted","Data":"42b65fd9e15426360d1e5dcd780ea1c525366e7a97bc43b3ddb77840abcc1960"}
Sep 17 23:09:00.755082 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:00.755033    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.738906224 podStartE2EDuration="2.755015052s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.108828ms" isStatefulPod=false podCreationTimestamp="2026-09-17 23:08:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:09:00.753946219 +0000 UTC m=+160.758138059" watchObservedRunningTime="2026-09-17 23:09:00.755015052 +0000 UTC m=+160.759206891"
Sep 17 23:09:13.857457 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.857421    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85fff65bf-vhhtb"]
Sep 17 23:09:13.860637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.860619    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:13.876580 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.876557    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85fff65bf-vhhtb"]
Sep 17 23:09:13.902717 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.902695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-oauth-config\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:13.902857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.902738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-trusted-ca-bundle\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:13.902857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.902762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-config\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:13.902857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.902780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-serving-cert\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:13.902857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.902836    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-oauth-serving-cert\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:13.903004 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.902902    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-service-ca\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:13.903004 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:13.902946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smhng\" (UniqueName: \"kubernetes.io/projected/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-kube-api-access-smhng\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.003817 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.003786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-oauth-serving-cert\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.003817 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.003817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-service-ca\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.004053 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.004017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smhng\" (UniqueName: \"kubernetes.io/projected/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-kube-api-access-smhng\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.004117 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.004086    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-oauth-config\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.004308 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.004283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-trusted-ca-bundle\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.004374 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.004327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-config\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.004442 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.004381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-serving-cert\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.004607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.004584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-service-ca\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.004962 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.004933    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-config\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.005080 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.004988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-oauth-serving-cert\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.005235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.005214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-trusted-ca-bundle\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.006656 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.006636    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-oauth-config\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.006834 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.006818    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-serving-cert\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.011562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.011536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smhng\" (UniqueName: \"kubernetes.io/projected/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-kube-api-access-smhng\") pod \"console-85fff65bf-vhhtb\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") " pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.169370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.169323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:14.442736 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.442708    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85fff65bf-vhhtb"]
Sep 17 23:09:14.446208 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:09:14.446180    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7fa44b7_d6ca_461f_8cc5_e8f10ff80228.slice/crio-4c24be6c546caf800e6843290e8a5e26c365240bc96710b735857d4c10e0a9ae WatchSource:0}: Error finding container 4c24be6c546caf800e6843290e8a5e26c365240bc96710b735857d4c10e0a9ae: Status 404 returned error can't find the container with id 4c24be6c546caf800e6843290e8a5e26c365240bc96710b735857d4c10e0a9ae
Sep 17 23:09:14.779226 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.779139    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85fff65bf-vhhtb" event={"ID":"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228","Type":"ContainerStarted","Data":"701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086"}
Sep 17 23:09:14.779226 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.779197    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85fff65bf-vhhtb" event={"ID":"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228","Type":"ContainerStarted","Data":"4c24be6c546caf800e6843290e8a5e26c365240bc96710b735857d4c10e0a9ae"}
Sep 17 23:09:14.797707 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:14.797662    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85fff65bf-vhhtb" podStartSLOduration=1.79764226 podStartE2EDuration="1.79764226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:09:14.79627763 +0000 UTC m=+174.800469468" watchObservedRunningTime="2026-09-17 23:09:14.79764226 +0000 UTC m=+174.801834098"
Sep 17 23:09:24.169775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:24.169735    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:24.170163 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:24.169787    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:24.174474 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:24.174449    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:24.817447 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:24.817420    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:09:24.863183 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:24.863148    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f564744c-dst2h"]
Sep 17 23:09:49.414744 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.414709    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"]
Sep 17 23:09:49.418192 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.418174    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.420776 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.420755    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:09:49.420776 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.420769    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zjqgk\""
Sep 17 23:09:49.421692 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.421677    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:09:49.427010 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.426984    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"]
Sep 17 23:09:49.590072 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.590034    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.590257 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.590083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.590257 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.590111    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcpll\" (UniqueName: \"kubernetes.io/projected/ceb0a8de-7149-499a-88a1-a19a2887023f-kube-api-access-tcpll\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.690799 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.690737    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcpll\" (UniqueName: \"kubernetes.io/projected/ceb0a8de-7149-499a-88a1-a19a2887023f-kube-api-access-tcpll\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.690867 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.690806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.690867 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.690832    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.691152 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.691116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.691196 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.691179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.699113 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.699085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcpll\" (UniqueName: \"kubernetes.io/projected/ceb0a8de-7149-499a-88a1-a19a2887023f-kube-api-access-tcpll\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.727031 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.727007    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:09:49.886771 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.886736    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64f564744c-dst2h" podUID="1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" containerName="console" containerID="cri-o://002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b" gracePeriod=15
Sep 17 23:09:49.995966 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:49.995943    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"]
Sep 17 23:09:49.997765 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:09:49.997738    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb0a8de_7149_499a_88a1_a19a2887023f.slice/crio-94320fa8c126192322e52bfa147df648c315f1324c1d6b9fa3f6a79d08d49c63 WatchSource:0}: Error finding container 94320fa8c126192322e52bfa147df648c315f1324c1d6b9fa3f6a79d08d49c63: Status 404 returned error can't find the container with id 94320fa8c126192322e52bfa147df648c315f1324c1d6b9fa3f6a79d08d49c63
Sep 17 23:09:50.114358 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.114334    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:09:50.195295 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195255    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-oauth-config\") pod \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") "
Sep 17 23:09:50.195462 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195302    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-serving-cert\") pod \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") "
Sep 17 23:09:50.195462 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195330    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-trusted-ca-bundle\") pod \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") "
Sep 17 23:09:50.195462 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195356    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-oauth-serving-cert\") pod \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") "
Sep 17 23:09:50.195600 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195577    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzjgp\" (UniqueName: \"kubernetes.io/projected/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-kube-api-access-bzjgp\") pod \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") "
Sep 17 23:09:50.195658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195639    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-config\") pod \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") "
Sep 17 23:09:50.195713 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195683    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-service-ca\") pod \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\" (UID: \"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a\") "
Sep 17 23:09:50.195835 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195806    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" (UID: "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:09:50.195835 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.195828    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" (UID: "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:09:50.196058 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.196040    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-trusted-ca-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:09:50.196144 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.196065    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-oauth-serving-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:09:50.196213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.196119    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-service-ca" (OuterVolumeSpecName: "service-ca") pod "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" (UID: "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:09:50.196213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.196145    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-config" (OuterVolumeSpecName: "console-config") pod "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" (UID: "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:09:50.197534 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.197512    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" (UID: "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:09:50.197627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.197537    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" (UID: "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:09:50.197627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.197593    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-kube-api-access-bzjgp" (OuterVolumeSpecName: "kube-api-access-bzjgp") pod "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" (UID: "1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a"). InnerVolumeSpecName "kube-api-access-bzjgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:09:50.296877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.296807    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzjgp\" (UniqueName: \"kubernetes.io/projected/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-kube-api-access-bzjgp\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:09:50.296877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.296832    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-config\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:09:50.296877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.296843    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:09:50.296877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.296851    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-oauth-config\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:09:50.296877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.296859    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a-console-serving-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:09:50.909263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.909228    2581 generic.go:356] "Generic (PLEG): container finished" podID="1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" containerID="002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b" exitCode=0
Sep 17 23:09:50.909729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.909277    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f564744c-dst2h" event={"ID":"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a","Type":"ContainerDied","Data":"002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b"}
Sep 17 23:09:50.909729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.909344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f564744c-dst2h" event={"ID":"1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a","Type":"ContainerDied","Data":"163c750f7d0403c6edba98c37e137c621baea8c1dc9f5552dc9c8b9fae98963c"}
Sep 17 23:09:50.909729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.909361    2581 scope.go:117] "RemoveContainer" containerID="002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b"
Sep 17 23:09:50.909729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.909406    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f564744c-dst2h"
Sep 17 23:09:50.910791 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.910751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t" event={"ID":"ceb0a8de-7149-499a-88a1-a19a2887023f","Type":"ContainerStarted","Data":"94320fa8c126192322e52bfa147df648c315f1324c1d6b9fa3f6a79d08d49c63"}
Sep 17 23:09:50.919277 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.919216    2581 scope.go:117] "RemoveContainer" containerID="002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b"
Sep 17 23:09:50.919634 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:09:50.919612    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b\": container with ID starting with 002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b not found: ID does not exist" containerID="002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b"
Sep 17 23:09:50.919730 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.919644    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b"} err="failed to get container status \"002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b\": rpc error: code = NotFound desc = could not find container \"002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b\": container with ID starting with 002ea29e252c4bc3fb10ef128b23b15f51551ecc48807ee1a86331e7138b401b not found: ID does not exist"
Sep 17 23:09:50.933062 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.932987    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f564744c-dst2h"]
Sep 17 23:09:50.934603 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:50.934581    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64f564744c-dst2h"]
Sep 17 23:09:51.992857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:51.992823    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" path="/var/lib/kubelet/pods/1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a/volumes"
Sep 17 23:09:55.934597 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:55.934563    2581 generic.go:356] "Generic (PLEG): container finished" podID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerID="9c92fe695f317ec5cc34d868f9e66abfef75f8da27e57bf3988a412459779ee8" exitCode=0
Sep 17 23:09:55.934955 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:55.934629    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t" event={"ID":"ceb0a8de-7149-499a-88a1-a19a2887023f","Type":"ContainerDied","Data":"9c92fe695f317ec5cc34d868f9e66abfef75f8da27e57bf3988a412459779ee8"}
Sep 17 23:09:58.949580 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:58.949541    2581 generic.go:356] "Generic (PLEG): container finished" podID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerID="80f982c076a90bef2a6b5058b0ca68ff3293f895c90d925484020fa5a6641edc" exitCode=0
Sep 17 23:09:58.949951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:09:58.949630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t" event={"ID":"ceb0a8de-7149-499a-88a1-a19a2887023f","Type":"ContainerDied","Data":"80f982c076a90bef2a6b5058b0ca68ff3293f895c90d925484020fa5a6641edc"}
Sep 17 23:10:05.979250 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:05.979218    2581 generic.go:356] "Generic (PLEG): container finished" podID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerID="611039068c4570c07c0fac6eab06d1ac6033903f396a6d5bdd353e3bca38929a" exitCode=0
Sep 17 23:10:05.979638 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:05.979276    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t" event={"ID":"ceb0a8de-7149-499a-88a1-a19a2887023f","Type":"ContainerDied","Data":"611039068c4570c07c0fac6eab06d1ac6033903f396a6d5bdd353e3bca38929a"}
Sep 17 23:10:07.108263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.108240    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:10:07.139376 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.139352    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-util\") pod \"ceb0a8de-7149-499a-88a1-a19a2887023f\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") "
Sep 17 23:10:07.139495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.139386    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcpll\" (UniqueName: \"kubernetes.io/projected/ceb0a8de-7149-499a-88a1-a19a2887023f-kube-api-access-tcpll\") pod \"ceb0a8de-7149-499a-88a1-a19a2887023f\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") "
Sep 17 23:10:07.139495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.139409    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-bundle\") pod \"ceb0a8de-7149-499a-88a1-a19a2887023f\" (UID: \"ceb0a8de-7149-499a-88a1-a19a2887023f\") "
Sep 17 23:10:07.140021 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.139996    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-bundle" (OuterVolumeSpecName: "bundle") pod "ceb0a8de-7149-499a-88a1-a19a2887023f" (UID: "ceb0a8de-7149-499a-88a1-a19a2887023f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:10:07.141531 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.141504    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb0a8de-7149-499a-88a1-a19a2887023f-kube-api-access-tcpll" (OuterVolumeSpecName: "kube-api-access-tcpll") pod "ceb0a8de-7149-499a-88a1-a19a2887023f" (UID: "ceb0a8de-7149-499a-88a1-a19a2887023f"). InnerVolumeSpecName "kube-api-access-tcpll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:10:07.143665 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.143644    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-util" (OuterVolumeSpecName: "util") pod "ceb0a8de-7149-499a-88a1-a19a2887023f" (UID: "ceb0a8de-7149-499a-88a1-a19a2887023f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:10:07.240919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.240852    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-util\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:07.240919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.240877    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcpll\" (UniqueName: \"kubernetes.io/projected/ceb0a8de-7149-499a-88a1-a19a2887023f-kube-api-access-tcpll\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:07.240919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.240888    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ceb0a8de-7149-499a-88a1-a19a2887023f-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:07.989196 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.989172    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t"
Sep 17 23:10:07.991911 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.991885    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shz4t" event={"ID":"ceb0a8de-7149-499a-88a1-a19a2887023f","Type":"ContainerDied","Data":"94320fa8c126192322e52bfa147df648c315f1324c1d6b9fa3f6a79d08d49c63"}
Sep 17 23:10:07.992040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:07.991921    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94320fa8c126192322e52bfa147df648c315f1324c1d6b9fa3f6a79d08d49c63"
Sep 17 23:10:12.128106 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128072    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"]
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128373    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" containerName="console"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128384    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" containerName="console"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128397    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerName="extract"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128403    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerName="extract"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128411    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerName="util"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128417    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerName="util"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128427    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerName="pull"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128432    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerName="pull"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128473    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a32d2fd-5b87-4b22-b61d-b9e8f7480b4a" containerName="console"
Sep 17 23:10:12.128552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.128480    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb0a8de-7149-499a-88a1-a19a2887023f" containerName="extract"
Sep 17 23:10:12.164063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.164033    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"]
Sep 17 23:10:12.164209 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.164146    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"
Sep 17 23:10:12.166811 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.166782    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 17 23:10:12.166955 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.166834    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lmbhp\""
Sep 17 23:10:12.166955 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.166834    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:10:12.176044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.176018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hltbm\" (UniqueName: \"kubernetes.io/projected/125a555e-c0bc-4ddd-bdfe-17be897e38a4-kube-api-access-hltbm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k2cj\" (UID: \"125a555e-c0bc-4ddd-bdfe-17be897e38a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"
Sep 17 23:10:12.176151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.176086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/125a555e-c0bc-4ddd-bdfe-17be897e38a4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k2cj\" (UID: \"125a555e-c0bc-4ddd-bdfe-17be897e38a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"
Sep 17 23:10:12.276789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.276749    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hltbm\" (UniqueName: \"kubernetes.io/projected/125a555e-c0bc-4ddd-bdfe-17be897e38a4-kube-api-access-hltbm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k2cj\" (UID: \"125a555e-c0bc-4ddd-bdfe-17be897e38a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"
Sep 17 23:10:12.276947 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.276839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/125a555e-c0bc-4ddd-bdfe-17be897e38a4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k2cj\" (UID: \"125a555e-c0bc-4ddd-bdfe-17be897e38a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"
Sep 17 23:10:12.277223 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.277204    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/125a555e-c0bc-4ddd-bdfe-17be897e38a4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k2cj\" (UID: \"125a555e-c0bc-4ddd-bdfe-17be897e38a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"
Sep 17 23:10:12.287722 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.287699    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hltbm\" (UniqueName: \"kubernetes.io/projected/125a555e-c0bc-4ddd-bdfe-17be897e38a4-kube-api-access-hltbm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k2cj\" (UID: \"125a555e-c0bc-4ddd-bdfe-17be897e38a4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"
Sep 17 23:10:12.473381 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.473298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"
Sep 17 23:10:12.771794 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:12.771770    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj"]
Sep 17 23:10:12.774025 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:10:12.774005    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125a555e_c0bc_4ddd_bdfe_17be897e38a4.slice/crio-d76ea7678a415b628abb9fd29f28abab56b876c9fa726af7a8971a99ce1cb4c3 WatchSource:0}: Error finding container d76ea7678a415b628abb9fd29f28abab56b876c9fa726af7a8971a99ce1cb4c3: Status 404 returned error can't find the container with id d76ea7678a415b628abb9fd29f28abab56b876c9fa726af7a8971a99ce1cb4c3
Sep 17 23:10:13.011158 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:13.011105    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj" event={"ID":"125a555e-c0bc-4ddd-bdfe-17be897e38a4","Type":"ContainerStarted","Data":"d76ea7678a415b628abb9fd29f28abab56b876c9fa726af7a8971a99ce1cb4c3"}
Sep 17 23:10:16.023763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.023726    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj" event={"ID":"125a555e-c0bc-4ddd-bdfe-17be897e38a4","Type":"ContainerStarted","Data":"3216197cb6c2fb2969e57c88405b600b849dd7eed9b2941a8c4068a51a24ebfd"}
Sep 17 23:10:16.044905 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.044858    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k2cj" podStartSLOduration=1.886350798 podStartE2EDuration="4.044842077s" totalImagesPullingTime="2.158491279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:10:16.043253718 +0000 UTC m=+236.047445556" watchObservedRunningTime="2026-09-17 23:10:16.044842077 +0000 UTC m=+236.049033916"
Sep 17 23:10:16.760715 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.760680    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"]
Sep 17 23:10:16.763221 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.763203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.765760 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.765737    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:10:16.765886 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.765773    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:10:16.766788 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.766768    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zjqgk\""
Sep 17 23:10:16.774277 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.774252    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"]
Sep 17 23:10:16.816599 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.816569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.816803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.816628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.816803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.816704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4qvn\" (UniqueName: \"kubernetes.io/projected/44349f92-fdea-4155-a956-cc7741a6e502-kube-api-access-b4qvn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.856812 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.856783    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-khg5t"]
Sep 17 23:10:16.858968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.858953    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:16.861750 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.861731    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 17 23:10:16.862085 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.862072    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-mm4t5\""
Sep 17 23:10:16.862343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.862330    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 17 23:10:16.871705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.871684    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-khg5t"]
Sep 17 23:10:16.917638 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.917606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.917757 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.917645    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9l2p\" (UniqueName: \"kubernetes.io/projected/ef87dfb1-f5d8-4d89-8855-60a3dd41e48b-kube-api-access-j9l2p\") pod \"cert-manager-webhook-6486ffd997-khg5t\" (UID: \"ef87dfb1-f5d8-4d89-8855-60a3dd41e48b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:16.917757 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.917680    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef87dfb1-f5d8-4d89-8855-60a3dd41e48b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-khg5t\" (UID: \"ef87dfb1-f5d8-4d89-8855-60a3dd41e48b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:16.917757 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.917706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4qvn\" (UniqueName: \"kubernetes.io/projected/44349f92-fdea-4155-a956-cc7741a6e502-kube-api-access-b4qvn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.917865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.917767    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.918060 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.918044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.918060 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.918053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:16.926251 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:16.926233    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4qvn\" (UniqueName: \"kubernetes.io/projected/44349f92-fdea-4155-a956-cc7741a6e502-kube-api-access-b4qvn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:17.018527 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:17.018457    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef87dfb1-f5d8-4d89-8855-60a3dd41e48b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-khg5t\" (UID: \"ef87dfb1-f5d8-4d89-8855-60a3dd41e48b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:17.018788 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:17.018756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9l2p\" (UniqueName: \"kubernetes.io/projected/ef87dfb1-f5d8-4d89-8855-60a3dd41e48b-kube-api-access-j9l2p\") pod \"cert-manager-webhook-6486ffd997-khg5t\" (UID: \"ef87dfb1-f5d8-4d89-8855-60a3dd41e48b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:17.028720 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:17.028695    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef87dfb1-f5d8-4d89-8855-60a3dd41e48b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-khg5t\" (UID: \"ef87dfb1-f5d8-4d89-8855-60a3dd41e48b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:17.029057 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:17.028998    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9l2p\" (UniqueName: \"kubernetes.io/projected/ef87dfb1-f5d8-4d89-8855-60a3dd41e48b-kube-api-access-j9l2p\") pod \"cert-manager-webhook-6486ffd997-khg5t\" (UID: \"ef87dfb1-f5d8-4d89-8855-60a3dd41e48b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:17.071972 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:17.071947    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:17.182035 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:17.182004    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:17.363386 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:17.363362    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"]
Sep 17 23:10:17.365428 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:10:17.365405    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44349f92_fdea_4155_a956_cc7741a6e502.slice/crio-5f249aeff3edd8c9a272a5466359bbd2d1bd29a690ee19de3947c3295adcf7a7 WatchSource:0}: Error finding container 5f249aeff3edd8c9a272a5466359bbd2d1bd29a690ee19de3947c3295adcf7a7: Status 404 returned error can't find the container with id 5f249aeff3edd8c9a272a5466359bbd2d1bd29a690ee19de3947c3295adcf7a7
Sep 17 23:10:17.449545 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:17.449519    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-khg5t"]
Sep 17 23:10:17.451173 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:10:17.451144    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef87dfb1_f5d8_4d89_8855_60a3dd41e48b.slice/crio-e74534394e3d74018aa67524871c7df72513597f2f085e59f44f913f9d04f09a WatchSource:0}: Error finding container e74534394e3d74018aa67524871c7df72513597f2f085e59f44f913f9d04f09a: Status 404 returned error can't find the container with id e74534394e3d74018aa67524871c7df72513597f2f085e59f44f913f9d04f09a
Sep 17 23:10:18.034631 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:18.034595    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t" event={"ID":"ef87dfb1-f5d8-4d89-8855-60a3dd41e48b","Type":"ContainerStarted","Data":"e74534394e3d74018aa67524871c7df72513597f2f085e59f44f913f9d04f09a"}
Sep 17 23:10:18.036309 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:18.036284    2581 generic.go:356] "Generic (PLEG): container finished" podID="44349f92-fdea-4155-a956-cc7741a6e502" containerID="68fcb04141f8c1b7a0ceee3a0ac5852169511a9ff3b6aa3432c89d80c9c1d27c" exitCode=0
Sep 17 23:10:18.036447 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:18.036368    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4" event={"ID":"44349f92-fdea-4155-a956-cc7741a6e502","Type":"ContainerDied","Data":"68fcb04141f8c1b7a0ceee3a0ac5852169511a9ff3b6aa3432c89d80c9c1d27c"}
Sep 17 23:10:18.036447 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:18.036420    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4" event={"ID":"44349f92-fdea-4155-a956-cc7741a6e502","Type":"ContainerStarted","Data":"5f249aeff3edd8c9a272a5466359bbd2d1bd29a690ee19de3947c3295adcf7a7"}
Sep 17 23:10:21.052081 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:21.051983    2581 generic.go:356] "Generic (PLEG): container finished" podID="44349f92-fdea-4155-a956-cc7741a6e502" containerID="49316b425adbc3673cd2d52796ffb412d89b6431a8002681b8beedb9ba5a7ca3" exitCode=0
Sep 17 23:10:21.052081 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:21.052062    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4" event={"ID":"44349f92-fdea-4155-a956-cc7741a6e502","Type":"ContainerDied","Data":"49316b425adbc3673cd2d52796ffb412d89b6431a8002681b8beedb9ba5a7ca3"}
Sep 17 23:10:21.053874 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:21.053850    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t" event={"ID":"ef87dfb1-f5d8-4d89-8855-60a3dd41e48b","Type":"ContainerStarted","Data":"b6e6fe124d018a32282747e0df960631dbd2a333a9ebd5d5d62cd3cf0da457ef"}
Sep 17 23:10:21.054003 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:21.053980    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:21.089228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:21.089186    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t" podStartSLOduration=1.810239886 podStartE2EDuration="5.089169635s" totalImagesPullingTime="3.278929749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:10:21.088014315 +0000 UTC m=+241.092206157" watchObservedRunningTime="2026-09-17 23:10:21.089169635 +0000 UTC m=+241.093361474"
Sep 17 23:10:22.059668 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:22.059637    2581 generic.go:356] "Generic (PLEG): container finished" podID="44349f92-fdea-4155-a956-cc7741a6e502" containerID="4a6ad32f8e3c61df9e9f26593747ac6bba0b7333a5de12fd1ae41850dc56e096" exitCode=0
Sep 17 23:10:22.060044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:22.059734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4" event={"ID":"44349f92-fdea-4155-a956-cc7741a6e502","Type":"ContainerDied","Data":"4a6ad32f8e3c61df9e9f26593747ac6bba0b7333a5de12fd1ae41850dc56e096"}
Sep 17 23:10:23.186281 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.186253    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:23.273770 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.273736    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4qvn\" (UniqueName: \"kubernetes.io/projected/44349f92-fdea-4155-a956-cc7741a6e502-kube-api-access-b4qvn\") pod \"44349f92-fdea-4155-a956-cc7741a6e502\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") "
Sep 17 23:10:23.273946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.273784    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-bundle\") pod \"44349f92-fdea-4155-a956-cc7741a6e502\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") "
Sep 17 23:10:23.273946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.273818    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-util\") pod \"44349f92-fdea-4155-a956-cc7741a6e502\" (UID: \"44349f92-fdea-4155-a956-cc7741a6e502\") "
Sep 17 23:10:23.274249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.274225    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-bundle" (OuterVolumeSpecName: "bundle") pod "44349f92-fdea-4155-a956-cc7741a6e502" (UID: "44349f92-fdea-4155-a956-cc7741a6e502"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:10:23.275916 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.275896    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44349f92-fdea-4155-a956-cc7741a6e502-kube-api-access-b4qvn" (OuterVolumeSpecName: "kube-api-access-b4qvn") pod "44349f92-fdea-4155-a956-cc7741a6e502" (UID: "44349f92-fdea-4155-a956-cc7741a6e502"). InnerVolumeSpecName "kube-api-access-b4qvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:10:23.278154 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.278106    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-util" (OuterVolumeSpecName: "util") pod "44349f92-fdea-4155-a956-cc7741a6e502" (UID: "44349f92-fdea-4155-a956-cc7741a6e502"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:10:23.374740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.374714    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-util\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:23.374740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.374740    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4qvn\" (UniqueName: \"kubernetes.io/projected/44349f92-fdea-4155-a956-cc7741a6e502-kube-api-access-b4qvn\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:23.374904 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:23.374751    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44349f92-fdea-4155-a956-cc7741a6e502-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:24.068908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:24.068871    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4" event={"ID":"44349f92-fdea-4155-a956-cc7741a6e502","Type":"ContainerDied","Data":"5f249aeff3edd8c9a272a5466359bbd2d1bd29a690ee19de3947c3295adcf7a7"}
Sep 17 23:10:24.069059 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:24.068920    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f249aeff3edd8c9a272a5466359bbd2d1bd29a690ee19de3947c3295adcf7a7"
Sep 17 23:10:24.069059 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:24.068881    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frktn4"
Sep 17 23:10:25.917142 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.917103    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8f6l9"]
Sep 17 23:10:25.917499 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.917447    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44349f92-fdea-4155-a956-cc7741a6e502" containerName="extract"
Sep 17 23:10:25.917499 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.917460    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="44349f92-fdea-4155-a956-cc7741a6e502" containerName="extract"
Sep 17 23:10:25.917499 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.917471    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44349f92-fdea-4155-a956-cc7741a6e502" containerName="util"
Sep 17 23:10:25.917499 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.917477    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="44349f92-fdea-4155-a956-cc7741a6e502" containerName="util"
Sep 17 23:10:25.917499 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.917489    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44349f92-fdea-4155-a956-cc7741a6e502" containerName="pull"
Sep 17 23:10:25.917499 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.917497    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="44349f92-fdea-4155-a956-cc7741a6e502" containerName="pull"
Sep 17 23:10:25.917713 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.917540    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="44349f92-fdea-4155-a956-cc7741a6e502" containerName="extract"
Sep 17 23:10:25.921404 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.921388    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8f6l9"
Sep 17 23:10:25.923823 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.923801    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-q6xfg\""
Sep 17 23:10:25.929037 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.929017    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8f6l9"]
Sep 17 23:10:25.996001 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.995973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88j9l\" (UniqueName: \"kubernetes.io/projected/2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96-kube-api-access-88j9l\") pod \"cert-manager-55b4d954d5-8f6l9\" (UID: \"2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96\") " pod="cert-manager/cert-manager-55b4d954d5-8f6l9"
Sep 17 23:10:25.996170 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:25.996018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96-bound-sa-token\") pod \"cert-manager-55b4d954d5-8f6l9\" (UID: \"2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96\") " pod="cert-manager/cert-manager-55b4d954d5-8f6l9"
Sep 17 23:10:26.097252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:26.097222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88j9l\" (UniqueName: \"kubernetes.io/projected/2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96-kube-api-access-88j9l\") pod \"cert-manager-55b4d954d5-8f6l9\" (UID: \"2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96\") " pod="cert-manager/cert-manager-55b4d954d5-8f6l9"
Sep 17 23:10:26.097382 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:26.097332    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96-bound-sa-token\") pod \"cert-manager-55b4d954d5-8f6l9\" (UID: \"2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96\") " pod="cert-manager/cert-manager-55b4d954d5-8f6l9"
Sep 17 23:10:26.105435 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:26.105413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96-bound-sa-token\") pod \"cert-manager-55b4d954d5-8f6l9\" (UID: \"2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96\") " pod="cert-manager/cert-manager-55b4d954d5-8f6l9"
Sep 17 23:10:26.105643 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:26.105621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88j9l\" (UniqueName: \"kubernetes.io/projected/2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96-kube-api-access-88j9l\") pod \"cert-manager-55b4d954d5-8f6l9\" (UID: \"2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96\") " pod="cert-manager/cert-manager-55b4d954d5-8f6l9"
Sep 17 23:10:26.230665 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:26.230565    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8f6l9"
Sep 17 23:10:26.503862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:26.503814    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8f6l9"]
Sep 17 23:10:26.505888 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:10:26.505861    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dbe4553_9c8e_4ff3_891f_aa9fff8a0a96.slice/crio-8e3f9ed61585ee29bae1b0b6029b83772caadf122c11a6bc65367bfa947514fd WatchSource:0}: Error finding container 8e3f9ed61585ee29bae1b0b6029b83772caadf122c11a6bc65367bfa947514fd: Status 404 returned error can't find the container with id 8e3f9ed61585ee29bae1b0b6029b83772caadf122c11a6bc65367bfa947514fd
Sep 17 23:10:27.062277 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:27.062248    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-khg5t"
Sep 17 23:10:27.082888 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:27.082855    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8f6l9" event={"ID":"2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96","Type":"ContainerStarted","Data":"59229c3a80c285c6e5a385f504ef1b5621840972d3930213670d59c4c1b04bd8"}
Sep 17 23:10:27.082888 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:27.082893    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8f6l9" event={"ID":"2dbe4553-9c8e-4ff3-891f-aa9fff8a0a96","Type":"ContainerStarted","Data":"8e3f9ed61585ee29bae1b0b6029b83772caadf122c11a6bc65367bfa947514fd"}
Sep 17 23:10:27.101916 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:27.101860    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-8f6l9" podStartSLOduration=2.101838423 podStartE2EDuration="2.101838423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:10:27.100963842 +0000 UTC m=+247.105155690" watchObservedRunningTime="2026-09-17 23:10:27.101838423 +0000 UTC m=+247.106030262"
Sep 17 23:10:43.415587 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.415505    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"]
Sep 17 23:10:43.418732 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.418714    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.421434 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.421407    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:10:43.421574 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.421531    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:10:43.422419 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.422403    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zjqgk\""
Sep 17 23:10:43.427109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.427088    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"]
Sep 17 23:10:43.538115 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.538079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.538303 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.538183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kl59\" (UniqueName: \"kubernetes.io/projected/7644d2dc-25b9-453e-864d-c7184845ee45-kube-api-access-2kl59\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.538303 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.538210    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.639558 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.639527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.639729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.639593    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kl59\" (UniqueName: \"kubernetes.io/projected/7644d2dc-25b9-453e-864d-c7184845ee45-kube-api-access-2kl59\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.639729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.639636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.639979 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.639957    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.640032 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.639996    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.649085 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.649052    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kl59\" (UniqueName: \"kubernetes.io/projected/7644d2dc-25b9-453e-864d-c7184845ee45-kube-api-access-2kl59\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.729009 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.728942    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:43.998482 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:43.998458    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"]
Sep 17 23:10:44.000342 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:10:44.000314    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7644d2dc_25b9_453e_864d_c7184845ee45.slice/crio-1ba0e2a5d4b6eb6145116a41871266c8af56a9070cc562f39da3f9539411952a WatchSource:0}: Error finding container 1ba0e2a5d4b6eb6145116a41871266c8af56a9070cc562f39da3f9539411952a: Status 404 returned error can't find the container with id 1ba0e2a5d4b6eb6145116a41871266c8af56a9070cc562f39da3f9539411952a
Sep 17 23:10:44.151159 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:44.151104    2581 generic.go:356] "Generic (PLEG): container finished" podID="7644d2dc-25b9-453e-864d-c7184845ee45" containerID="29a9ba1f15ff2c9034dab3438f791582306dbe7783d33e4e35ad7d1b427644c2" exitCode=0
Sep 17 23:10:44.151286 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:44.151153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw" event={"ID":"7644d2dc-25b9-453e-864d-c7184845ee45","Type":"ContainerDied","Data":"29a9ba1f15ff2c9034dab3438f791582306dbe7783d33e4e35ad7d1b427644c2"}
Sep 17 23:10:44.151286 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:44.151228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw" event={"ID":"7644d2dc-25b9-453e-864d-c7184845ee45","Type":"ContainerStarted","Data":"1ba0e2a5d4b6eb6145116a41871266c8af56a9070cc562f39da3f9539411952a"}
Sep 17 23:10:45.157592 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:45.157560    2581 generic.go:356] "Generic (PLEG): container finished" podID="7644d2dc-25b9-453e-864d-c7184845ee45" containerID="56e9331859d1992232684b9d46be7d58bfc04f8932e3d3d82a6bea227258f7b5" exitCode=0
Sep 17 23:10:45.157976 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:45.157631    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw" event={"ID":"7644d2dc-25b9-453e-864d-c7184845ee45","Type":"ContainerDied","Data":"56e9331859d1992232684b9d46be7d58bfc04f8932e3d3d82a6bea227258f7b5"}
Sep 17 23:10:46.164496 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:46.164461    2581 generic.go:356] "Generic (PLEG): container finished" podID="7644d2dc-25b9-453e-864d-c7184845ee45" containerID="05aa713aad74d5033182f82f884a4002dee13090060814af92dec534fffa1b90" exitCode=0
Sep 17 23:10:46.164853 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:46.164545    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw" event={"ID":"7644d2dc-25b9-453e-864d-c7184845ee45","Type":"ContainerDied","Data":"05aa713aad74d5033182f82f884a4002dee13090060814af92dec534fffa1b90"}
Sep 17 23:10:47.292162 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.292140    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:47.370115 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.370082    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-util\") pod \"7644d2dc-25b9-453e-864d-c7184845ee45\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") "
Sep 17 23:10:47.370115 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.370140    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-bundle\") pod \"7644d2dc-25b9-453e-864d-c7184845ee45\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") "
Sep 17 23:10:47.370329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.370223    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kl59\" (UniqueName: \"kubernetes.io/projected/7644d2dc-25b9-453e-864d-c7184845ee45-kube-api-access-2kl59\") pod \"7644d2dc-25b9-453e-864d-c7184845ee45\" (UID: \"7644d2dc-25b9-453e-864d-c7184845ee45\") "
Sep 17 23:10:47.370878 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.370847    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-bundle" (OuterVolumeSpecName: "bundle") pod "7644d2dc-25b9-453e-864d-c7184845ee45" (UID: "7644d2dc-25b9-453e-864d-c7184845ee45"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:10:47.372190 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.372167    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7644d2dc-25b9-453e-864d-c7184845ee45-kube-api-access-2kl59" (OuterVolumeSpecName: "kube-api-access-2kl59") pod "7644d2dc-25b9-453e-864d-c7184845ee45" (UID: "7644d2dc-25b9-453e-864d-c7184845ee45"). InnerVolumeSpecName "kube-api-access-2kl59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:10:47.375569 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.375532    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-util" (OuterVolumeSpecName: "util") pod "7644d2dc-25b9-453e-864d-c7184845ee45" (UID: "7644d2dc-25b9-453e-864d-c7184845ee45"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:10:47.471740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.471647    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kl59\" (UniqueName: \"kubernetes.io/projected/7644d2dc-25b9-453e-864d-c7184845ee45-kube-api-access-2kl59\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:47.471740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.471691    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-util\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:47.471740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:47.471706    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7644d2dc-25b9-453e-864d-c7184845ee45-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:48.175635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:48.175584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw" event={"ID":"7644d2dc-25b9-453e-864d-c7184845ee45","Type":"ContainerDied","Data":"1ba0e2a5d4b6eb6145116a41871266c8af56a9070cc562f39da3f9539411952a"}
Sep 17 23:10:48.175635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:48.175621    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afwzkhw"
Sep 17 23:10:48.175838 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:48.175651    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ba0e2a5d4b6eb6145116a41871266c8af56a9070cc562f39da3f9539411952a"
Sep 17 23:10:53.629386 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.629347    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"]
Sep 17 23:10:53.630567 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.630540    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7644d2dc-25b9-453e-864d-c7184845ee45" containerName="extract"
Sep 17 23:10:53.630724 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.630713    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7644d2dc-25b9-453e-864d-c7184845ee45" containerName="extract"
Sep 17 23:10:53.630808 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.630799    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7644d2dc-25b9-453e-864d-c7184845ee45" containerName="util"
Sep 17 23:10:53.630878 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.630870    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7644d2dc-25b9-453e-864d-c7184845ee45" containerName="util"
Sep 17 23:10:53.630948 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.630939    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7644d2dc-25b9-453e-864d-c7184845ee45" containerName="pull"
Sep 17 23:10:53.631017 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.631010    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7644d2dc-25b9-453e-864d-c7184845ee45" containerName="pull"
Sep 17 23:10:53.631266 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.631250    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7644d2dc-25b9-453e-864d-c7184845ee45" containerName="extract"
Sep 17 23:10:53.677348 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.677311    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"]
Sep 17 23:10:53.677480 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.677411    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.680053 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.680023    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:10:53.681174 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.681152    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:10:53.681288 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.681228    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zjqgk\""
Sep 17 23:10:53.822365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.822327    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbr47\" (UniqueName: \"kubernetes.io/projected/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-kube-api-access-bbr47\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.822511 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.822369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.822511 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.822397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.923756 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.923682    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbr47\" (UniqueName: \"kubernetes.io/projected/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-kube-api-access-bbr47\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.923756 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.923720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.923756 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.923739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.924092 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.924072    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.924158 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.924094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.934075 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.934052    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbr47\" (UniqueName: \"kubernetes.io/projected/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-kube-api-access-bbr47\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:53.986888 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:53.986854    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:54.257134 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:54.257097    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"]
Sep 17 23:10:54.259472 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:10:54.259443    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee4c9bf_f6ff_40e9_8246_a081bdd6aca4.slice/crio-b2ff9f3857565fa8e32d8cd5471c6b7732c610cae3dc235ed89807cef8af9786 WatchSource:0}: Error finding container b2ff9f3857565fa8e32d8cd5471c6b7732c610cae3dc235ed89807cef8af9786: Status 404 returned error can't find the container with id b2ff9f3857565fa8e32d8cd5471c6b7732c610cae3dc235ed89807cef8af9786
Sep 17 23:10:55.207975 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:55.207895    2581 generic.go:356] "Generic (PLEG): container finished" podID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerID="ccb46a625c0cffa9b4a12df3dad33d0f603b13b5cca4319641e5fa5fadc732b0" exitCode=0
Sep 17 23:10:55.208325 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:55.207973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs" event={"ID":"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4","Type":"ContainerDied","Data":"ccb46a625c0cffa9b4a12df3dad33d0f603b13b5cca4319641e5fa5fadc732b0"}
Sep 17 23:10:55.208325 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:55.208027    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs" event={"ID":"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4","Type":"ContainerStarted","Data":"b2ff9f3857565fa8e32d8cd5471c6b7732c610cae3dc235ed89807cef8af9786"}
Sep 17 23:10:56.039299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.039272    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"]
Sep 17 23:10:56.042495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.042475    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.045519 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.045500    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 17 23:10:56.045639 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.045542    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lj7s7\""
Sep 17 23:10:56.045639 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.045578    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 17 23:10:56.046058 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.046040    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 17 23:10:56.046120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.046043    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 17 23:10:56.061970 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.061946    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"]
Sep 17 23:10:56.144073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.143994    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/237e6004-a756-4f77-b9b4-783adee35380-manifests\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.144073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.144024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/237e6004-a756-4f77-b9b4-783adee35380-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.144073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.144052    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcxjx\" (UniqueName: \"kubernetes.io/projected/237e6004-a756-4f77-b9b4-783adee35380-kube-api-access-pcxjx\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.144334 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.144297    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/237e6004-a756-4f77-b9b4-783adee35380-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.144452 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.144429    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/237e6004-a756-4f77-b9b4-783adee35380-tmp\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.214551 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.214521    2581 generic.go:356] "Generic (PLEG): container finished" podID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerID="32d36e8c5ebc75abb259a6dc719ab312f10dc158d2915a65286d74b219a56b9b" exitCode=0
Sep 17 23:10:56.214917 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.214560    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs" event={"ID":"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4","Type":"ContainerDied","Data":"32d36e8c5ebc75abb259a6dc719ab312f10dc158d2915a65286d74b219a56b9b"}
Sep 17 23:10:56.246108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.245677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/237e6004-a756-4f77-b9b4-783adee35380-manifests\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.246108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.245717    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/237e6004-a756-4f77-b9b4-783adee35380-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.246108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.245745    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcxjx\" (UniqueName: \"kubernetes.io/projected/237e6004-a756-4f77-b9b4-783adee35380-kube-api-access-pcxjx\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.246108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.245812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/237e6004-a756-4f77-b9b4-783adee35380-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.246108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.245866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/237e6004-a756-4f77-b9b4-783adee35380-tmp\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.246108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.246064    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/237e6004-a756-4f77-b9b4-783adee35380-manifests\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.248187 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.248163    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/237e6004-a756-4f77-b9b4-783adee35380-tmp\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.248308 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.248290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/237e6004-a756-4f77-b9b4-783adee35380-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.248391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.248374    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/237e6004-a756-4f77-b9b4-783adee35380-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.254178 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.254151    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcxjx\" (UniqueName: \"kubernetes.io/projected/237e6004-a756-4f77-b9b4-783adee35380-kube-api-access-pcxjx\") pod \"opendatahub-operator-controller-manager-5d5976b594-lltk9\" (UID: \"237e6004-a756-4f77-b9b4-783adee35380\") " pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.352274 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.352240    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:10:56.630996 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:56.630970    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"]
Sep 17 23:10:56.633422 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:10:56.633395    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237e6004_a756_4f77_b9b4_783adee35380.slice/crio-9088f9bda49fda39c7e8c1ddb53c2b7cd2b8547179f3d434d6b3f80fc4c44e3e WatchSource:0}: Error finding container 9088f9bda49fda39c7e8c1ddb53c2b7cd2b8547179f3d434d6b3f80fc4c44e3e: Status 404 returned error can't find the container with id 9088f9bda49fda39c7e8c1ddb53c2b7cd2b8547179f3d434d6b3f80fc4c44e3e
Sep 17 23:10:57.220814 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:57.220729    2581 generic.go:356] "Generic (PLEG): container finished" podID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerID="89affca0a93f4ec5d09382ecf621245b805e6ecea7dcc8006b009cdb9ba32e50" exitCode=0
Sep 17 23:10:57.220814 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:57.220796    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs" event={"ID":"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4","Type":"ContainerDied","Data":"89affca0a93f4ec5d09382ecf621245b805e6ecea7dcc8006b009cdb9ba32e50"}
Sep 17 23:10:57.222160 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:57.222140    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9" event={"ID":"237e6004-a756-4f77-b9b4-783adee35380","Type":"ContainerStarted","Data":"9088f9bda49fda39c7e8c1ddb53c2b7cd2b8547179f3d434d6b3f80fc4c44e3e"}
Sep 17 23:10:59.679959 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.679926    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:10:59.774096 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.774073    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-util\") pod \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") "
Sep 17 23:10:59.774212 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.774175    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbr47\" (UniqueName: \"kubernetes.io/projected/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-kube-api-access-bbr47\") pod \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") "
Sep 17 23:10:59.774275 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.774251    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-bundle\") pod \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\" (UID: \"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4\") "
Sep 17 23:10:59.775288 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.775203    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-bundle" (OuterVolumeSpecName: "bundle") pod "cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" (UID: "cee4c9bf-f6ff-40e9-8246-a081bdd6aca4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:10:59.776742 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.776718    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-kube-api-access-bbr47" (OuterVolumeSpecName: "kube-api-access-bbr47") pod "cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" (UID: "cee4c9bf-f6ff-40e9-8246-a081bdd6aca4"). InnerVolumeSpecName "kube-api-access-bbr47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:10:59.780511 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.780491    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-util" (OuterVolumeSpecName: "util") pod "cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" (UID: "cee4c9bf-f6ff-40e9-8246-a081bdd6aca4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:10:59.874826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.874791    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:59.874826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.874818    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-util\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:10:59.874826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:10:59.874828    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbr47\" (UniqueName: \"kubernetes.io/projected/cee4c9bf-f6ff-40e9-8246-a081bdd6aca4-kube-api-access-bbr47\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:11:00.240096 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:00.240066    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs"
Sep 17 23:11:00.240272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:00.240066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nt5zs" event={"ID":"cee4c9bf-f6ff-40e9-8246-a081bdd6aca4","Type":"ContainerDied","Data":"b2ff9f3857565fa8e32d8cd5471c6b7732c610cae3dc235ed89807cef8af9786"}
Sep 17 23:11:00.240272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:00.240245    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2ff9f3857565fa8e32d8cd5471c6b7732c610cae3dc235ed89807cef8af9786"
Sep 17 23:11:00.241823 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:00.241792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9" event={"ID":"237e6004-a756-4f77-b9b4-783adee35380","Type":"ContainerStarted","Data":"c3e30880e95c612c7dfcd2dfa9558f6abd7a809e9da150ff0dcc30d963e276bb"}
Sep 17 23:11:01.247880 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:01.247789    2581 generic.go:356] "Generic (PLEG): container finished" podID="237e6004-a756-4f77-b9b4-783adee35380" containerID="c3e30880e95c612c7dfcd2dfa9558f6abd7a809e9da150ff0dcc30d963e276bb" exitCode=0
Sep 17 23:11:01.247880 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:01.247839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9" event={"ID":"237e6004-a756-4f77-b9b4-783adee35380","Type":"ContainerDied","Data":"c3e30880e95c612c7dfcd2dfa9558f6abd7a809e9da150ff0dcc30d963e276bb"}
Sep 17 23:11:02.254631 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:02.254594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9" event={"ID":"237e6004-a756-4f77-b9b4-783adee35380","Type":"ContainerStarted","Data":"3f201c6b044cd763f1d22515f15410829da1ecc087f9ba20ee88c746f7f81ce5"}
Sep 17 23:11:02.254993 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:02.254711    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:11:02.279876 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:02.279827    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9" podStartSLOduration=1.706701941 podStartE2EDuration="6.279812362s" totalImagesPullingTime="3.514011468s" totalInitContainerRuntime="1.059098953s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:10:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:02.277630876 +0000 UTC m=+282.281822708" watchObservedRunningTime="2026-09-17 23:11:02.279812362 +0000 UTC m=+282.284004202"
Sep 17 23:11:13.261316 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.261285    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d5976b594-lltk9"
Sep 17 23:11:13.413458 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.413425    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"]
Sep 17 23:11:13.413739 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.413727    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerName="util"
Sep 17 23:11:13.413775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.413741    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerName="util"
Sep 17 23:11:13.413775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.413750    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerName="extract"
Sep 17 23:11:13.413775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.413755    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerName="extract"
Sep 17 23:11:13.413884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.413778    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerName="pull"
Sep 17 23:11:13.413884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.413786    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerName="pull"
Sep 17 23:11:13.413884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.413844    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cee4c9bf-f6ff-40e9-8246-a081bdd6aca4" containerName="extract"
Sep 17 23:11:13.420599 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.420580    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.423044 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.423001    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 17 23:11:13.423220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.423076    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 17 23:11:13.423220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.423077    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 17 23:11:13.423627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.423422    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 17 23:11:13.423627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.423428    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-zfkq4\""
Sep 17 23:11:13.423627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.423490    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 17 23:11:13.425546 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.425526    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"]
Sep 17 23:11:13.473771 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.473743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-manager-config\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.473909 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.473781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-cert\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.474011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.473984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-metrics-cert\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.474074 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.474055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjpbv\" (UniqueName: \"kubernetes.io/projected/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-kube-api-access-tjpbv\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.575111 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.575031    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-manager-config\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.575111 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.575077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-cert\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.575111 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.575110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-metrics-cert\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.575324 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.575147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjpbv\" (UniqueName: \"kubernetes.io/projected/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-kube-api-access-tjpbv\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.575837 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.575807    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-manager-config\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.577554 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.577536    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-metrics-cert\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.577650 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.577582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-cert\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.583560 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.583539    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjpbv\" (UniqueName: \"kubernetes.io/projected/bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677-kube-api-access-tjpbv\") pod \"lws-controller-manager-ccddb67c-f8ggm\" (UID: \"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677\") " pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:13.731218 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:13.731188    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:14.010276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:14.010253    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"]
Sep 17 23:11:14.012414 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:11:14.012387    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb6ce8a8_3e24_4c1d_aba1_b974b2cc0677.slice/crio-fbae9669643a72d2612cc4249f19fc4ec6924a7b1bb807e4bb8a0c65caca0c7f WatchSource:0}: Error finding container fbae9669643a72d2612cc4249f19fc4ec6924a7b1bb807e4bb8a0c65caca0c7f: Status 404 returned error can't find the container with id fbae9669643a72d2612cc4249f19fc4ec6924a7b1bb807e4bb8a0c65caca0c7f
Sep 17 23:11:14.308214 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:14.308119    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm" event={"ID":"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677","Type":"ContainerStarted","Data":"fbae9669643a72d2612cc4249f19fc4ec6924a7b1bb807e4bb8a0c65caca0c7f"}
Sep 17 23:11:16.326984 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:16.326942    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm" event={"ID":"bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677","Type":"ContainerStarted","Data":"6adb7540a213ed770ac014553ca1abc4a2a9968c9c5dac82a235e14a944de9b2"}
Sep 17 23:11:16.327433 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:16.327104    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:16.351222 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:16.351177    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm" podStartSLOduration=2.061073271 podStartE2EDuration="3.351162066s" totalImagesPullingTime="1.290088795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:16.348581773 +0000 UTC m=+296.352773612" watchObservedRunningTime="2026-09-17 23:11:16.351162066 +0000 UTC m=+296.355353906"
Sep 17 23:11:21.898379 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:21.898352    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 23:11:22.971060 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:22.971029    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"]
Sep 17 23:11:22.974311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:22.974291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:22.977446 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:22.977405    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lmjwf\""
Sep 17 23:11:22.977627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:22.977479    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 23:11:22.977627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:22.977520    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 23:11:22.977777 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:22.977762    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 23:11:22.985385 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:22.985359    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"]
Sep 17 23:11:23.058481 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058449    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.058481 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.058718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.058718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.058718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.058718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3336558e-734f-49fb-b82a-0e05a2a2a582-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.058718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.058941 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.058941 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.058800    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8d5g\" (UniqueName: \"kubernetes.io/projected/3336558e-734f-49fb-b82a-0e05a2a2a582-kube-api-access-c8d5g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.159678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.159678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.159910 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159715    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.159910 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159739    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.159910 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159773    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3336558e-734f-49fb-b82a-0e05a2a2a582-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.159910 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.159910 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.159910 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8d5g\" (UniqueName: \"kubernetes.io/projected/3336558e-734f-49fb-b82a-0e05a2a2a582-kube-api-access-c8d5g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.160239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.159975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.160239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.160151    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.160239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.160188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.160345 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.160296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.160568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.160546    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3336558e-734f-49fb-b82a-0e05a2a2a582-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.160646 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.160542    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.162208 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.162187    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.162569 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.162548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.168079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.168053    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3336558e-734f-49fb-b82a-0e05a2a2a582-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.168217 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.168200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8d5g\" (UniqueName: \"kubernetes.io/projected/3336558e-734f-49fb-b82a-0e05a2a2a582-kube-api-access-c8d5g\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq\" (UID: \"3336558e-734f-49fb-b82a-0e05a2a2a582\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.285214 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.285142    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:23.566017 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.565966    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"]
Sep 17 23:11:23.568154 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:11:23.568111    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3336558e_734f_49fb_b82a_0e05a2a2a582.slice/crio-062a84ba97962fa9eeafbb5632ebc32dc406184c03c90998ebf8f5b0f2cc8a7e WatchSource:0}: Error finding container 062a84ba97962fa9eeafbb5632ebc32dc406184c03c90998ebf8f5b0f2cc8a7e: Status 404 returned error can't find the container with id 062a84ba97962fa9eeafbb5632ebc32dc406184c03c90998ebf8f5b0f2cc8a7e
Sep 17 23:11:23.570408 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:23.570393    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:11:24.367764 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:24.367714    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq" event={"ID":"3336558e-734f-49fb-b82a-0e05a2a2a582","Type":"ContainerStarted","Data":"062a84ba97962fa9eeafbb5632ebc32dc406184c03c90998ebf8f5b0f2cc8a7e"}
Sep 17 23:11:25.960950 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:25.960917    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r48zj"]
Sep 17 23:11:25.969235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:25.969212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:25.971954 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:25.971768    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xd55m\""
Sep 17 23:11:25.972317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:25.972154    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 17 23:11:25.975306 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:25.975278    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r48zj"]
Sep 17 23:11:26.085709 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.085669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/359bfa42-ebaf-4f98-b787-6e024edd5b4b-cert\") pod \"odh-model-controller-7998c67dc4-r48zj\" (UID: \"359bfa42-ebaf-4f98-b787-6e024edd5b4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:26.085901 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.085736    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vn2h\" (UniqueName: \"kubernetes.io/projected/359bfa42-ebaf-4f98-b787-6e024edd5b4b-kube-api-access-9vn2h\") pod \"odh-model-controller-7998c67dc4-r48zj\" (UID: \"359bfa42-ebaf-4f98-b787-6e024edd5b4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:26.186480 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.186444    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/359bfa42-ebaf-4f98-b787-6e024edd5b4b-cert\") pod \"odh-model-controller-7998c67dc4-r48zj\" (UID: \"359bfa42-ebaf-4f98-b787-6e024edd5b4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:26.186480 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.186487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vn2h\" (UniqueName: \"kubernetes.io/projected/359bfa42-ebaf-4f98-b787-6e024edd5b4b-kube-api-access-9vn2h\") pod \"odh-model-controller-7998c67dc4-r48zj\" (UID: \"359bfa42-ebaf-4f98-b787-6e024edd5b4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:26.186731 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:11:26.186626    2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 17 23:11:26.186731 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:11:26.186721    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/359bfa42-ebaf-4f98-b787-6e024edd5b4b-cert podName:359bfa42-ebaf-4f98-b787-6e024edd5b4b nodeName:}" failed. No retries permitted until 2026-09-17 23:11:26.686699031 +0000 UTC m=+306.690890889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/359bfa42-ebaf-4f98-b787-6e024edd5b4b-cert") pod "odh-model-controller-7998c67dc4-r48zj" (UID: "359bfa42-ebaf-4f98-b787-6e024edd5b4b") : secret "odh-model-controller-webhook-cert" not found
Sep 17 23:11:26.195763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.195745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vn2h\" (UniqueName: \"kubernetes.io/projected/359bfa42-ebaf-4f98-b787-6e024edd5b4b-kube-api-access-9vn2h\") pod \"odh-model-controller-7998c67dc4-r48zj\" (UID: \"359bfa42-ebaf-4f98-b787-6e024edd5b4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:26.197081 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.197053    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:11:26.197157 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.197144    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:11:26.197192 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.197175    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:11:26.380513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.380474    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq" event={"ID":"3336558e-734f-49fb-b82a-0e05a2a2a582","Type":"ContainerStarted","Data":"1cad258a4556747fb4bcf41b4fdec73f3782e608e144f883fd9ecc27631fcdef"}
Sep 17 23:11:26.404245 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.404184    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq" podStartSLOduration=1.777863237 podStartE2EDuration="4.404164526s" totalImagesPullingTime="2.626301289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:26.402151765 +0000 UTC m=+306.406343598" watchObservedRunningTime="2026-09-17 23:11:26.404164526 +0000 UTC m=+306.408356366"
Sep 17 23:11:26.548763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.548692    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v5lxw"]
Sep 17 23:11:26.552017 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.551998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:26.554494 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.554470    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 17 23:11:26.564045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.564021    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v5lxw"]
Sep 17 23:11:26.691453 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.691418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/359bfa42-ebaf-4f98-b787-6e024edd5b4b-cert\") pod \"odh-model-controller-7998c67dc4-r48zj\" (UID: \"359bfa42-ebaf-4f98-b787-6e024edd5b4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:26.691453 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.691457    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84hf8\" (UniqueName: \"kubernetes.io/projected/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-kube-api-access-84hf8\") pod \"model-serving-api-54d89bd5fc-v5lxw\" (UID: \"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:26.691672 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.691484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-tls-certs\") pod \"model-serving-api-54d89bd5fc-v5lxw\" (UID: \"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:26.693818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.693793    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/359bfa42-ebaf-4f98-b787-6e024edd5b4b-cert\") pod \"odh-model-controller-7998c67dc4-r48zj\" (UID: \"359bfa42-ebaf-4f98-b787-6e024edd5b4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:26.792032 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.791995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84hf8\" (UniqueName: \"kubernetes.io/projected/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-kube-api-access-84hf8\") pod \"model-serving-api-54d89bd5fc-v5lxw\" (UID: \"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:26.792032 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.792039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-tls-certs\") pod \"model-serving-api-54d89bd5fc-v5lxw\" (UID: \"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:26.792314 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:11:26.792166    2581 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 17 23:11:26.792314 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:11:26.792223    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-tls-certs podName:42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1 nodeName:}" failed. No retries permitted until 2026-09-17 23:11:27.292207894 +0000 UTC m=+307.296399724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-tls-certs") pod "model-serving-api-54d89bd5fc-v5lxw" (UID: "42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1") : secret "model-serving-api-tls" not found
Sep 17 23:11:26.800512 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.800453    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84hf8\" (UniqueName: \"kubernetes.io/projected/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-kube-api-access-84hf8\") pod \"model-serving-api-54d89bd5fc-v5lxw\" (UID: \"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:26.882481 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:26.882447    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:27.154353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.154331    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-r48zj"]
Sep 17 23:11:27.156677 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:11:27.156652    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod359bfa42_ebaf_4f98_b787_6e024edd5b4b.slice/crio-aeb5f59738805cf7f3e5ee3dc85f7a73b8080083a0b526fccb698187ef67df1a WatchSource:0}: Error finding container aeb5f59738805cf7f3e5ee3dc85f7a73b8080083a0b526fccb698187ef67df1a: Status 404 returned error can't find the container with id aeb5f59738805cf7f3e5ee3dc85f7a73b8080083a0b526fccb698187ef67df1a
Sep 17 23:11:27.286106 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.286076    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:27.290673 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.290650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:27.296960 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.296941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-tls-certs\") pod \"model-serving-api-54d89bd5fc-v5lxw\" (UID: \"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:27.299228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.299206    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1-tls-certs\") pod \"model-serving-api-54d89bd5fc-v5lxw\" (UID: \"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1\") " pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:27.335200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.335177    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-ccddb67c-f8ggm"
Sep 17 23:11:27.387173 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.387138    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r48zj" event={"ID":"359bfa42-ebaf-4f98-b787-6e024edd5b4b","Type":"ContainerStarted","Data":"aeb5f59738805cf7f3e5ee3dc85f7a73b8080083a0b526fccb698187ef67df1a"}
Sep 17 23:11:27.387420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.387395    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:27.388495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.388476    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq"
Sep 17 23:11:27.462820 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.462740    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:27.738635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:27.738553    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v5lxw"]
Sep 17 23:11:27.743883 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:11:27.743857    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a82f45_f8f2_4ffd_baf8_e4ec2d4564a1.slice/crio-ed9fb7fc3dd773c1215f7bcafe9770da69583a5802683cf5f480c6ccef1726f6 WatchSource:0}: Error finding container ed9fb7fc3dd773c1215f7bcafe9770da69583a5802683cf5f480c6ccef1726f6: Status 404 returned error can't find the container with id ed9fb7fc3dd773c1215f7bcafe9770da69583a5802683cf5f480c6ccef1726f6
Sep 17 23:11:28.400164 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:28.400103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw" event={"ID":"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1","Type":"ContainerStarted","Data":"ed9fb7fc3dd773c1215f7bcafe9770da69583a5802683cf5f480c6ccef1726f6"}
Sep 17 23:11:32.422011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.421971    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-r48zj" event={"ID":"359bfa42-ebaf-4f98-b787-6e024edd5b4b","Type":"ContainerStarted","Data":"1e919c01375d7576f8ad8b0e057a4b2f8e5a5aae571eb9b50390e2366f283d1d"}
Sep 17 23:11:32.422476 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.422132    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:32.423447 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.423425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw" event={"ID":"42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1","Type":"ContainerStarted","Data":"22bffa159fd13fb75ecd7f5fcbd67c92767d6f339b9082cbd60afbb87fe3159f"}
Sep 17 23:11:32.423568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.423485    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:11:32.447904 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.447854    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-r48zj" podStartSLOduration=3.119452024 podStartE2EDuration="7.447838257s" totalImagesPullingTime="4.328386233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:32.446452086 +0000 UTC m=+312.450643926" watchObservedRunningTime="2026-09-17 23:11:32.447838257 +0000 UTC m=+312.452030096"
Sep 17 23:11:32.467525 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.467472    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw" podStartSLOduration=2.68937677 podStartE2EDuration="6.467456632s" totalImagesPullingTime="3.778079862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:32.466812304 +0000 UTC m=+312.471004143" watchObservedRunningTime="2026-09-17 23:11:32.467456632 +0000 UTC m=+312.471648472"
Sep 17 23:11:32.630525 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.630483    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"]
Sep 17 23:11:32.633986 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.633964    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.636455 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.636431    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 17 23:11:32.636568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.636455    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zz26r\""
Sep 17 23:11:32.636718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.636699    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 17 23:11:32.636797 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.636783    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 17 23:11:32.649293 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.649269    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"]
Sep 17 23:11:32.746005 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.745927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb7vj\" (UniqueName: \"kubernetes.io/projected/7732a4eb-adc8-4011-af97-586a4bf96319-kube-api-access-wb7vj\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.746005 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.745973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.746261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.746007    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7732a4eb-adc8-4011-af97-586a4bf96319-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.746261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.746083    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7732a4eb-adc8-4011-af97-586a4bf96319-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.746261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.746173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7732a4eb-adc8-4011-af97-586a4bf96319-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.746261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.746243    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.746456 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.746275    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.846720 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.846684    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb7vj\" (UniqueName: \"kubernetes.io/projected/7732a4eb-adc8-4011-af97-586a4bf96319-kube-api-access-wb7vj\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.846884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.846746    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.846884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.846787    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7732a4eb-adc8-4011-af97-586a4bf96319-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.846884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.846819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7732a4eb-adc8-4011-af97-586a4bf96319-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.846884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.846861    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7732a4eb-adc8-4011-af97-586a4bf96319-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.847093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.846928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.847093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.846958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.847672 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.847646    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7732a4eb-adc8-4011-af97-586a4bf96319-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.849331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.849302    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.849428 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.849358    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.849619 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.849600    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7732a4eb-adc8-4011-af97-586a4bf96319-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.849661 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.849615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7732a4eb-adc8-4011-af97-586a4bf96319-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.862621 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.862589    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7732a4eb-adc8-4011-af97-586a4bf96319-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.862773 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.862757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb7vj\" (UniqueName: \"kubernetes.io/projected/7732a4eb-adc8-4011-af97-586a4bf96319-kube-api-access-wb7vj\") pod \"istiod-openshift-gateway-5677ccb49b-crrwm\" (UID: \"7732a4eb-adc8-4011-af97-586a4bf96319\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:32.943051 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:32.943024    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:33.235436 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:33.235408    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"]
Sep 17 23:11:33.236287 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:11:33.236266    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7732a4eb_adc8_4011_af97_586a4bf96319.slice/crio-550a30e74016d11bdd02af3e3bab110a8747f5495bc811f7e2638b36ad232dd8 WatchSource:0}: Error finding container 550a30e74016d11bdd02af3e3bab110a8747f5495bc811f7e2638b36ad232dd8: Status 404 returned error can't find the container with id 550a30e74016d11bdd02af3e3bab110a8747f5495bc811f7e2638b36ad232dd8
Sep 17 23:11:33.429298 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:33.429257    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm" event={"ID":"7732a4eb-adc8-4011-af97-586a4bf96319","Type":"ContainerStarted","Data":"550a30e74016d11bdd02af3e3bab110a8747f5495bc811f7e2638b36ad232dd8"}
Sep 17 23:11:35.355370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:35.355330    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:11:35.355639 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:35.355400    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:11:35.442198 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:35.442158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm" event={"ID":"7732a4eb-adc8-4011-af97-586a4bf96319","Type":"ContainerStarted","Data":"bb886a00c9fda31ae2012ebaf5807c47167ea667775500efaa21c80633ad7542"}
Sep 17 23:11:35.442354 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:35.442234    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:35.510080 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:35.510024    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm" podStartSLOduration=1.393326155 podStartE2EDuration="3.510006071s" totalImagesPullingTime="2.116679916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:11:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:11:35.507865859 +0000 UTC m=+315.512057699" watchObservedRunningTime="2026-09-17 23:11:35.510006071 +0000 UTC m=+315.514197909"
Sep 17 23:11:36.449052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:36.449017    2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-crrwm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 23:11:36.449515 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:36.449076    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm" podUID="7732a4eb-adc8-4011-af97-586a4bf96319" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:11:39.448339 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:39.448309    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-crrwm"
Sep 17 23:11:43.431832 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:43.431800    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-r48zj"
Sep 17 23:11:43.433573 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:11:43.433553    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-v5lxw"
Sep 17 23:12:30.837674 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.837639    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"]
Sep 17 23:12:30.844165 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.844147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:30.847022 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.847001    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 17 23:12:30.847022 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.847024    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 17 23:12:30.848350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.848327    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zjqgk\""
Sep 17 23:12:30.852017 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.851851    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"]
Sep 17 23:12:30.940114 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.940084    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"]
Sep 17 23:12:30.943513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.943497    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:30.952273 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.952248    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"]
Sep 17 23:12:30.952694 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.952668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:30.952778 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.952720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwzt7\" (UniqueName: \"kubernetes.io/projected/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-kube-api-access-dwzt7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:30.952778 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:30.952761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:31.034799 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.034764    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"]
Sep 17 23:12:31.038228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.038213    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.048169 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.048141    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"]
Sep 17 23:12:31.053777 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.053754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jstx\" (UniqueName: \"kubernetes.io/projected/511e07fe-963c-47c9-b597-a6627827c2cc-kube-api-access-7jstx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.053912 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.053788    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:31.053912 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.053807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.053912 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.053907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwzt7\" (UniqueName: \"kubernetes.io/projected/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-kube-api-access-dwzt7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:31.054079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.053931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:31.054079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.053949    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.054232 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.054208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:31.054280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.054247    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:31.062199 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.062180    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwzt7\" (UniqueName: \"kubernetes.io/projected/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-kube-api-access-dwzt7\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:31.138694 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.138655    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"]
Sep 17 23:12:31.142305 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.142289    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.150915 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.150893    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"]
Sep 17 23:12:31.154707 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.154683    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:31.154936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.154913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jstx\" (UniqueName: \"kubernetes.io/projected/511e07fe-963c-47c9-b597-a6627827c2cc-kube-api-access-7jstx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.155011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.154949    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.155011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.154969    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.155011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.154998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.155207 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.155032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.155207 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.155053    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnz8m\" (UniqueName: \"kubernetes.io/projected/18def50b-ff74-450f-bb1b-c9ccac2766f2-kube-api-access-cnz8m\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.155344 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.155321    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.155403 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.155353    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.166061 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.166032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jstx\" (UniqueName: \"kubernetes.io/projected/511e07fe-963c-47c9-b597-a6627827c2cc-kube-api-access-7jstx\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.253468 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.253438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:31.255835 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.255799    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.255955 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.255838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.255955 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.255895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j7zk\" (UniqueName: \"kubernetes.io/projected/811944dc-24d9-4962-978f-1166b21106aa-kube-api-access-4j7zk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.256025 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.255991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.256063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.256024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnz8m\" (UniqueName: \"kubernetes.io/projected/18def50b-ff74-450f-bb1b-c9ccac2766f2-kube-api-access-cnz8m\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.256063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.256050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.256295 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.256274    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.256338 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.256300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.264294 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.264267    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnz8m\" (UniqueName: \"kubernetes.io/projected/18def50b-ff74-450f-bb1b-c9ccac2766f2-kube-api-access-cnz8m\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.347857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.347812    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:31.357217 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.357197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.357310 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.357254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.357310 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.357296    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j7zk\" (UniqueName: \"kubernetes.io/projected/811944dc-24d9-4962-978f-1166b21106aa-kube-api-access-4j7zk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.357624 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.357606    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.357691 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.357641    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.365750 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.365713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j7zk\" (UniqueName: \"kubernetes.io/projected/811944dc-24d9-4962-978f-1166b21106aa-kube-api-access-4j7zk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.430065 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.430038    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"]
Sep 17 23:12:31.432158 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:12:31.432116    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4cab88_55ed_4145_bc0c_d32ba0aca01f.slice/crio-1a65ff64470c663add67828642ab50f60eda6e0af5ac647ac3ca5576550ff720 WatchSource:0}: Error finding container 1a65ff64470c663add67828642ab50f60eda6e0af5ac647ac3ca5576550ff720: Status 404 returned error can't find the container with id 1a65ff64470c663add67828642ab50f60eda6e0af5ac647ac3ca5576550ff720
Sep 17 23:12:31.452479 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.452455    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:31.529303 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.528858    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"]
Sep 17 23:12:31.531031 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:12:31.531002    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511e07fe_963c_47c9_b597_a6627827c2cc.slice/crio-55589aa6601febb7d887b3dfd1e5adf68fd9e9d6150cd5119a0e991214390eec WatchSource:0}: Error finding container 55589aa6601febb7d887b3dfd1e5adf68fd9e9d6150cd5119a0e991214390eec: Status 404 returned error can't find the container with id 55589aa6601febb7d887b3dfd1e5adf68fd9e9d6150cd5119a0e991214390eec
Sep 17 23:12:31.621167 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.621116    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"]
Sep 17 23:12:31.627964 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:12:31.627941    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18def50b_ff74_450f_bb1b_c9ccac2766f2.slice/crio-4b5183d2933efa94efc411c8bc2bff623b3806f1ccfcf46216ca77cb4578b994 WatchSource:0}: Error finding container 4b5183d2933efa94efc411c8bc2bff623b3806f1ccfcf46216ca77cb4578b994: Status 404 returned error can't find the container with id 4b5183d2933efa94efc411c8bc2bff623b3806f1ccfcf46216ca77cb4578b994
Sep 17 23:12:31.697678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.697650    2581 generic.go:356] "Generic (PLEG): container finished" podID="511e07fe-963c-47c9-b597-a6627827c2cc" containerID="30e686054630f09bc80e43041cc1cd4696932ba6b461a783a5dd5a18af12ebc2" exitCode=0
Sep 17 23:12:31.697795 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.697734    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26" event={"ID":"511e07fe-963c-47c9-b597-a6627827c2cc","Type":"ContainerDied","Data":"30e686054630f09bc80e43041cc1cd4696932ba6b461a783a5dd5a18af12ebc2"}
Sep 17 23:12:31.697872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.697793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26" event={"ID":"511e07fe-963c-47c9-b597-a6627827c2cc","Type":"ContainerStarted","Data":"55589aa6601febb7d887b3dfd1e5adf68fd9e9d6150cd5119a0e991214390eec"}
Sep 17 23:12:31.699400 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.699378    2581 generic.go:356] "Generic (PLEG): container finished" podID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerID="97a1086fe0ea32ff8a123fd7f5e8e4c1e589e36c0b969b1672e049f3cc75be9d" exitCode=0
Sep 17 23:12:31.699485 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.699425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2" event={"ID":"ea4cab88-55ed-4145-bc0c-d32ba0aca01f","Type":"ContainerDied","Data":"97a1086fe0ea32ff8a123fd7f5e8e4c1e589e36c0b969b1672e049f3cc75be9d"}
Sep 17 23:12:31.699485 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.699464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2" event={"ID":"ea4cab88-55ed-4145-bc0c-d32ba0aca01f","Type":"ContainerStarted","Data":"1a65ff64470c663add67828642ab50f60eda6e0af5ac647ac3ca5576550ff720"}
Sep 17 23:12:31.701533 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.701427    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb" event={"ID":"18def50b-ff74-450f-bb1b-c9ccac2766f2","Type":"ContainerStarted","Data":"6541c0b6d42269d323ff58e77e94ff264cf007cc222fe7d08a43bfa86b202c51"}
Sep 17 23:12:31.701533 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.701472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb" event={"ID":"18def50b-ff74-450f-bb1b-c9ccac2766f2","Type":"ContainerStarted","Data":"4b5183d2933efa94efc411c8bc2bff623b3806f1ccfcf46216ca77cb4578b994"}
Sep 17 23:12:31.729494 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:31.729347    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"]
Sep 17 23:12:31.734202 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:12:31.734174    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811944dc_24d9_4962_978f_1166b21106aa.slice/crio-de79a39e656edb4d5ed463521f32515ed7e20a8a5c7adda5ee8246a8e7c61acd WatchSource:0}: Error finding container de79a39e656edb4d5ed463521f32515ed7e20a8a5c7adda5ee8246a8e7c61acd: Status 404 returned error can't find the container with id de79a39e656edb4d5ed463521f32515ed7e20a8a5c7adda5ee8246a8e7c61acd
Sep 17 23:12:32.707984 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:32.707949    2581 generic.go:356] "Generic (PLEG): container finished" podID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerID="6541c0b6d42269d323ff58e77e94ff264cf007cc222fe7d08a43bfa86b202c51" exitCode=0
Sep 17 23:12:32.708410 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:32.708037    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb" event={"ID":"18def50b-ff74-450f-bb1b-c9ccac2766f2","Type":"ContainerDied","Data":"6541c0b6d42269d323ff58e77e94ff264cf007cc222fe7d08a43bfa86b202c51"}
Sep 17 23:12:32.709623 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:32.709601    2581 generic.go:356] "Generic (PLEG): container finished" podID="811944dc-24d9-4962-978f-1166b21106aa" containerID="106a0f4a7f17b8a1fab174ab5e202cbf7c9b724e8fa1f463e2ccd49d8bd4cb71" exitCode=0
Sep 17 23:12:32.709720 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:32.709698    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x" event={"ID":"811944dc-24d9-4962-978f-1166b21106aa","Type":"ContainerDied","Data":"106a0f4a7f17b8a1fab174ab5e202cbf7c9b724e8fa1f463e2ccd49d8bd4cb71"}
Sep 17 23:12:32.709845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:32.709740    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x" event={"ID":"811944dc-24d9-4962-978f-1166b21106aa","Type":"ContainerStarted","Data":"de79a39e656edb4d5ed463521f32515ed7e20a8a5c7adda5ee8246a8e7c61acd"}
Sep 17 23:12:33.716675 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:33.716640    2581 generic.go:356] "Generic (PLEG): container finished" podID="511e07fe-963c-47c9-b597-a6627827c2cc" containerID="2503fc21c254a5076775fcb3270de5236eb8a4f7a367a4e8647312fd2ad0ebc5" exitCode=0
Sep 17 23:12:33.717100 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:33.716723    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26" event={"ID":"511e07fe-963c-47c9-b597-a6627827c2cc","Type":"ContainerDied","Data":"2503fc21c254a5076775fcb3270de5236eb8a4f7a367a4e8647312fd2ad0ebc5"}
Sep 17 23:12:33.718642 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:33.718622    2581 generic.go:356] "Generic (PLEG): container finished" podID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerID="e04c997f841bb2fab0758f5b4cfb232237004ec61a34c50beb9ccc8d9349e6ca" exitCode=0
Sep 17 23:12:33.718715 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:33.718656    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2" event={"ID":"ea4cab88-55ed-4145-bc0c-d32ba0aca01f","Type":"ContainerDied","Data":"e04c997f841bb2fab0758f5b4cfb232237004ec61a34c50beb9ccc8d9349e6ca"}
Sep 17 23:12:34.725869 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:34.725833    2581 generic.go:356] "Generic (PLEG): container finished" podID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerID="687032b03e7dea7fb324d4c24a7b40d6784d2f9e2a1669ff3ec7e8f093f810b6" exitCode=0
Sep 17 23:12:34.726316 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:34.725920    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb" event={"ID":"18def50b-ff74-450f-bb1b-c9ccac2766f2","Type":"ContainerDied","Data":"687032b03e7dea7fb324d4c24a7b40d6784d2f9e2a1669ff3ec7e8f093f810b6"}
Sep 17 23:12:34.728165 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:34.728141    2581 generic.go:356] "Generic (PLEG): container finished" podID="511e07fe-963c-47c9-b597-a6627827c2cc" containerID="09ae7fb79ae584773300c362c609d7c314f19b092c9021672599b657229b0a20" exitCode=0
Sep 17 23:12:34.728240 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:34.728214    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26" event={"ID":"511e07fe-963c-47c9-b597-a6627827c2cc","Type":"ContainerDied","Data":"09ae7fb79ae584773300c362c609d7c314f19b092c9021672599b657229b0a20"}
Sep 17 23:12:34.730319 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:34.730301    2581 generic.go:356] "Generic (PLEG): container finished" podID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerID="8dd4bf2710375a417b4593e069068c9a8670676b1451a3cf8a4e86bbc89b0062" exitCode=0
Sep 17 23:12:34.730407 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:34.730358    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2" event={"ID":"ea4cab88-55ed-4145-bc0c-d32ba0aca01f","Type":"ContainerDied","Data":"8dd4bf2710375a417b4593e069068c9a8670676b1451a3cf8a4e86bbc89b0062"}
Sep 17 23:12:34.732330 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:34.732311    2581 generic.go:356] "Generic (PLEG): container finished" podID="811944dc-24d9-4962-978f-1166b21106aa" containerID="8e6741df26aae16e7b99cb9e72223d15c15b031935d9fb75bd8bf0f7e9887c49" exitCode=0
Sep 17 23:12:34.732409 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:34.732346    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x" event={"ID":"811944dc-24d9-4962-978f-1166b21106aa","Type":"ContainerDied","Data":"8e6741df26aae16e7b99cb9e72223d15c15b031935d9fb75bd8bf0f7e9887c49"}
Sep 17 23:12:35.739866 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:35.739831    2581 generic.go:356] "Generic (PLEG): container finished" podID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerID="13e40afccdefb0656dd568680caa69082ceda8dd2b6a4f0eb17fd564f47d8fb0" exitCode=0
Sep 17 23:12:35.740326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:35.739917    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb" event={"ID":"18def50b-ff74-450f-bb1b-c9ccac2766f2","Type":"ContainerDied","Data":"13e40afccdefb0656dd568680caa69082ceda8dd2b6a4f0eb17fd564f47d8fb0"}
Sep 17 23:12:35.742066 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:35.742046    2581 generic.go:356] "Generic (PLEG): container finished" podID="811944dc-24d9-4962-978f-1166b21106aa" containerID="69638a839f727161a791cd86fc8c1d659ccab2014c25ba0d038f4de5dee5d03b" exitCode=0
Sep 17 23:12:35.742188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:35.742137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x" event={"ID":"811944dc-24d9-4962-978f-1166b21106aa","Type":"ContainerDied","Data":"69638a839f727161a791cd86fc8c1d659ccab2014c25ba0d038f4de5dee5d03b"}
Sep 17 23:12:35.868400 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:35.868376    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:35.902315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:35.902290    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:36.002387 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.002308    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-util\") pod \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") "
Sep 17 23:12:36.002387 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.002367    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwzt7\" (UniqueName: \"kubernetes.io/projected/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-kube-api-access-dwzt7\") pod \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") "
Sep 17 23:12:36.002632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.002399    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-util\") pod \"511e07fe-963c-47c9-b597-a6627827c2cc\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") "
Sep 17 23:12:36.002632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.002462    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-bundle\") pod \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\" (UID: \"ea4cab88-55ed-4145-bc0c-d32ba0aca01f\") "
Sep 17 23:12:36.002632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.002504    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-bundle\") pod \"511e07fe-963c-47c9-b597-a6627827c2cc\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") "
Sep 17 23:12:36.002632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.002542    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jstx\" (UniqueName: \"kubernetes.io/projected/511e07fe-963c-47c9-b597-a6627827c2cc-kube-api-access-7jstx\") pod \"511e07fe-963c-47c9-b597-a6627827c2cc\" (UID: \"511e07fe-963c-47c9-b597-a6627827c2cc\") "
Sep 17 23:12:36.003043 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.003004    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-bundle" (OuterVolumeSpecName: "bundle") pod "ea4cab88-55ed-4145-bc0c-d32ba0aca01f" (UID: "ea4cab88-55ed-4145-bc0c-d32ba0aca01f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:36.003173 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.003101    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-bundle" (OuterVolumeSpecName: "bundle") pod "511e07fe-963c-47c9-b597-a6627827c2cc" (UID: "511e07fe-963c-47c9-b597-a6627827c2cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:36.004676 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.004649    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511e07fe-963c-47c9-b597-a6627827c2cc-kube-api-access-7jstx" (OuterVolumeSpecName: "kube-api-access-7jstx") pod "511e07fe-963c-47c9-b597-a6627827c2cc" (UID: "511e07fe-963c-47c9-b597-a6627827c2cc"). InnerVolumeSpecName "kube-api-access-7jstx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:12:36.005011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.004988    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-kube-api-access-dwzt7" (OuterVolumeSpecName: "kube-api-access-dwzt7") pod "ea4cab88-55ed-4145-bc0c-d32ba0aca01f" (UID: "ea4cab88-55ed-4145-bc0c-d32ba0aca01f"). InnerVolumeSpecName "kube-api-access-dwzt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:12:36.008548 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.008524    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-util" (OuterVolumeSpecName: "util") pod "511e07fe-963c-47c9-b597-a6627827c2cc" (UID: "511e07fe-963c-47c9-b597-a6627827c2cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:36.009033 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.009014    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-util" (OuterVolumeSpecName: "util") pod "ea4cab88-55ed-4145-bc0c-d32ba0aca01f" (UID: "ea4cab88-55ed-4145-bc0c-d32ba0aca01f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:36.103404 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.103374    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:36.103404 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.103400    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:36.103404 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.103411    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jstx\" (UniqueName: \"kubernetes.io/projected/511e07fe-963c-47c9-b597-a6627827c2cc-kube-api-access-7jstx\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:36.103621 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.103421    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-util\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:36.103621 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.103429    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwzt7\" (UniqueName: \"kubernetes.io/projected/ea4cab88-55ed-4145-bc0c-d32ba0aca01f-kube-api-access-dwzt7\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:36.103621 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.103436    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/511e07fe-963c-47c9-b597-a6627827c2cc-util\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:36.748830 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.748796    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26" event={"ID":"511e07fe-963c-47c9-b597-a6627827c2cc","Type":"ContainerDied","Data":"55589aa6601febb7d887b3dfd1e5adf68fd9e9d6150cd5119a0e991214390eec"}
Sep 17 23:12:36.749262 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.748847    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55589aa6601febb7d887b3dfd1e5adf68fd9e9d6150cd5119a0e991214390eec"
Sep 17 23:12:36.749262 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.748806    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cvdx26"
Sep 17 23:12:36.750715 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.750698    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2"
Sep 17 23:12:36.750845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.750727    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tqhk2" event={"ID":"ea4cab88-55ed-4145-bc0c-d32ba0aca01f","Type":"ContainerDied","Data":"1a65ff64470c663add67828642ab50f60eda6e0af5ac647ac3ca5576550ff720"}
Sep 17 23:12:36.750845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.750765    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a65ff64470c663add67828642ab50f60eda6e0af5ac647ac3ca5576550ff720"
Sep 17 23:12:36.879741 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.879716    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:36.912237 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:36.912214    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:37.009203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.009117    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-bundle\") pod \"18def50b-ff74-450f-bb1b-c9ccac2766f2\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") "
Sep 17 23:12:37.009203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.009187    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnz8m\" (UniqueName: \"kubernetes.io/projected/18def50b-ff74-450f-bb1b-c9ccac2766f2-kube-api-access-cnz8m\") pod \"18def50b-ff74-450f-bb1b-c9ccac2766f2\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") "
Sep 17 23:12:37.009416 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.009205    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-util\") pod \"811944dc-24d9-4962-978f-1166b21106aa\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") "
Sep 17 23:12:37.009416 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.009234    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-bundle\") pod \"811944dc-24d9-4962-978f-1166b21106aa\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") "
Sep 17 23:12:37.009416 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.009265    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-util\") pod \"18def50b-ff74-450f-bb1b-c9ccac2766f2\" (UID: \"18def50b-ff74-450f-bb1b-c9ccac2766f2\") "
Sep 17 23:12:37.009416 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.009304    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j7zk\" (UniqueName: \"kubernetes.io/projected/811944dc-24d9-4962-978f-1166b21106aa-kube-api-access-4j7zk\") pod \"811944dc-24d9-4962-978f-1166b21106aa\" (UID: \"811944dc-24d9-4962-978f-1166b21106aa\") "
Sep 17 23:12:37.009766 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.009744    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-bundle" (OuterVolumeSpecName: "bundle") pod "18def50b-ff74-450f-bb1b-c9ccac2766f2" (UID: "18def50b-ff74-450f-bb1b-c9ccac2766f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:37.009973 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.009949    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-bundle" (OuterVolumeSpecName: "bundle") pod "811944dc-24d9-4962-978f-1166b21106aa" (UID: "811944dc-24d9-4962-978f-1166b21106aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:37.011509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.011487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811944dc-24d9-4962-978f-1166b21106aa-kube-api-access-4j7zk" (OuterVolumeSpecName: "kube-api-access-4j7zk") pod "811944dc-24d9-4962-978f-1166b21106aa" (UID: "811944dc-24d9-4962-978f-1166b21106aa"). InnerVolumeSpecName "kube-api-access-4j7zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:12:37.011608 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.011515    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18def50b-ff74-450f-bb1b-c9ccac2766f2-kube-api-access-cnz8m" (OuterVolumeSpecName: "kube-api-access-cnz8m") pod "18def50b-ff74-450f-bb1b-c9ccac2766f2" (UID: "18def50b-ff74-450f-bb1b-c9ccac2766f2"). InnerVolumeSpecName "kube-api-access-cnz8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:12:37.110434 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.110400    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j7zk\" (UniqueName: \"kubernetes.io/projected/811944dc-24d9-4962-978f-1166b21106aa-kube-api-access-4j7zk\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:37.110434 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.110428    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:37.110434 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.110440    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnz8m\" (UniqueName: \"kubernetes.io/projected/18def50b-ff74-450f-bb1b-c9ccac2766f2-kube-api-access-cnz8m\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:37.110644 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.110451    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:37.141847 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.141808    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-util" (OuterVolumeSpecName: "util") pod "18def50b-ff74-450f-bb1b-c9ccac2766f2" (UID: "18def50b-ff74-450f-bb1b-c9ccac2766f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:37.144842 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.144816    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-util" (OuterVolumeSpecName: "util") pod "811944dc-24d9-4962-978f-1166b21106aa" (UID: "811944dc-24d9-4962-978f-1166b21106aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:12:37.211235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.211204    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/811944dc-24d9-4962-978f-1166b21106aa-util\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:37.211235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.211229    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/18def50b-ff74-450f-bb1b-c9ccac2766f2-util\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:12:37.758079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.758048    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb"
Sep 17 23:12:37.758079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.758062    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7htnvb" event={"ID":"18def50b-ff74-450f-bb1b-c9ccac2766f2","Type":"ContainerDied","Data":"4b5183d2933efa94efc411c8bc2bff623b3806f1ccfcf46216ca77cb4578b994"}
Sep 17 23:12:37.758630 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.758102    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b5183d2933efa94efc411c8bc2bff623b3806f1ccfcf46216ca77cb4578b994"
Sep 17 23:12:37.759980 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.759962    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x"
Sep 17 23:12:37.760136 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.759962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847aj4f8x" event={"ID":"811944dc-24d9-4962-978f-1166b21106aa","Type":"ContainerDied","Data":"de79a39e656edb4d5ed463521f32515ed7e20a8a5c7adda5ee8246a8e7c61acd"}
Sep 17 23:12:37.760136 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:37.760064    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de79a39e656edb4d5ed463521f32515ed7e20a8a5c7adda5ee8246a8e7c61acd"
Sep 17 23:12:39.899212 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:12:39.899175    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85fff65bf-vhhtb"]
Sep 17 23:13:04.918010 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:04.917974    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85fff65bf-vhhtb" podUID="f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" containerName="console" containerID="cri-o://701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086" gracePeriod=15
Sep 17 23:13:05.158471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.158443    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:13:05.261299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261206    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-oauth-config\") pod \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") "
Sep 17 23:13:05.261299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261256    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-config\") pod \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") "
Sep 17 23:13:05.261299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261280    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-serving-cert\") pod \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") "
Sep 17 23:13:05.261542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261312    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smhng\" (UniqueName: \"kubernetes.io/projected/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-kube-api-access-smhng\") pod \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") "
Sep 17 23:13:05.261542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261388    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-service-ca\") pod \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") "
Sep 17 23:13:05.261542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261420    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-trusted-ca-bundle\") pod \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") "
Sep 17 23:13:05.261542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261458    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-oauth-serving-cert\") pod \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\" (UID: \"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228\") "
Sep 17 23:13:05.261804 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261759    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-config" (OuterVolumeSpecName: "console-config") pod "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" (UID: "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:13:05.261804 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261793    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" (UID: "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:13:05.261939 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.261770    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-service-ca" (OuterVolumeSpecName: "service-ca") pod "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" (UID: "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:13:05.262055 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.262018    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" (UID: "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:13:05.263395 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.263375    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" (UID: "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:13:05.263484 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.263472    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" (UID: "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:13:05.263572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.263554    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-kube-api-access-smhng" (OuterVolumeSpecName: "kube-api-access-smhng") pod "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" (UID: "f7fa44b7-d6ca-461f-8cc5-e8f10ff80228"). InnerVolumeSpecName "kube-api-access-smhng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:13:05.362772 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.362737    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-trusted-ca-bundle\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:13:05.362772 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.362764    2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-oauth-serving-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:13:05.362772 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.362776    2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-oauth-config\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:13:05.362999 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.362788    2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-config\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:13:05.362999 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.362799    2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-console-serving-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:13:05.362999 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.362812    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smhng\" (UniqueName: \"kubernetes.io/projected/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-kube-api-access-smhng\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:13:05.362999 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.362825    2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:13:05.904382 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.904349    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" containerID="701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086" exitCode=0
Sep 17 23:13:05.904582 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.904390    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85fff65bf-vhhtb"
Sep 17 23:13:05.904582 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.904432    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85fff65bf-vhhtb" event={"ID":"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228","Type":"ContainerDied","Data":"701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086"}
Sep 17 23:13:05.904582 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.904484    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85fff65bf-vhhtb" event={"ID":"f7fa44b7-d6ca-461f-8cc5-e8f10ff80228","Type":"ContainerDied","Data":"4c24be6c546caf800e6843290e8a5e26c365240bc96710b735857d4c10e0a9ae"}
Sep 17 23:13:05.904582 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.904506    2581 scope.go:117] "RemoveContainer" containerID="701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086"
Sep 17 23:13:05.913689 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.913672    2581 scope.go:117] "RemoveContainer" containerID="701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086"
Sep 17 23:13:05.913952 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:13:05.913933    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086\": container with ID starting with 701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086 not found: ID does not exist" containerID="701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086"
Sep 17 23:13:05.914001 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.913961    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086"} err="failed to get container status \"701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086\": rpc error: code = NotFound desc = could not find container \"701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086\": container with ID starting with 701db4f2791bd3e4b2e52136c5e63a241d3acd01e9bd78c7ad6cc599f7fb8086 not found: ID does not exist"
Sep 17 23:13:05.938857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.938828    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85fff65bf-vhhtb"]
Sep 17 23:13:05.944498 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.944477    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85fff65bf-vhhtb"]
Sep 17 23:13:05.999054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:05.999020    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" path="/var/lib/kubelet/pods/f7fa44b7-d6ca-461f-8cc5-e8f10ff80228/volumes"
Sep 17 23:13:39.444153 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444066    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"]
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444393    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerName="extract"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444409    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerName="extract"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444421    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811944dc-24d9-4962-978f-1166b21106aa" containerName="pull"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444428    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="811944dc-24d9-4962-978f-1166b21106aa" containerName="pull"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444434    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerName="pull"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444439    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerName="pull"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444446    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811944dc-24d9-4962-978f-1166b21106aa" containerName="extract"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444451    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="811944dc-24d9-4962-978f-1166b21106aa" containerName="extract"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444463    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511e07fe-963c-47c9-b597-a6627827c2cc" containerName="pull"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444469    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="511e07fe-963c-47c9-b597-a6627827c2cc" containerName="pull"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444477    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerName="extract"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444482    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerName="extract"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444489    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511e07fe-963c-47c9-b597-a6627827c2cc" containerName="extract"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444494    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="511e07fe-963c-47c9-b597-a6627827c2cc" containerName="extract"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444502    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerName="util"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444507    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerName="util"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444516    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511e07fe-963c-47c9-b597-a6627827c2cc" containerName="util"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444520    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="511e07fe-963c-47c9-b597-a6627827c2cc" containerName="util"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444527    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerName="pull"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444533    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerName="pull"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444540    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerName="util"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444545    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerName="util"
Sep 17 23:13:39.444540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444550    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811944dc-24d9-4962-978f-1166b21106aa" containerName="util"
Sep 17 23:13:39.445213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444555    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="811944dc-24d9-4962-978f-1166b21106aa" containerName="util"
Sep 17 23:13:39.445213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444561    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" containerName="console"
Sep 17 23:13:39.445213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444566    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" containerName="console"
Sep 17 23:13:39.445213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444612    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="18def50b-ff74-450f-bb1b-c9ccac2766f2" containerName="extract"
Sep 17 23:13:39.445213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444619    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4cab88-55ed-4145-bc0c-d32ba0aca01f" containerName="extract"
Sep 17 23:13:39.445213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444627    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fa44b7-d6ca-461f-8cc5-e8f10ff80228" containerName="console"
Sep 17 23:13:39.445213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444634    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="511e07fe-963c-47c9-b597-a6627827c2cc" containerName="extract"
Sep 17 23:13:39.445213 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.444642    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="811944dc-24d9-4962-978f-1166b21106aa" containerName="extract"
Sep 17 23:13:39.447727 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.447707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.450702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.450682    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2zq8t\""
Sep 17 23:13:39.464060 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.464029    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"]
Sep 17 23:13:39.548896 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.548862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/615d0917-5b05-44ad-856a-f7e723ba954d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.549056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.548902    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/615d0917-5b05-44ad-856a-f7e723ba954d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.549056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.548929    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.549056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.548954    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.549056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.548975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/615d0917-5b05-44ad-856a-f7e723ba954d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.549056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.549006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.549260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.549076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.549260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.549112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knvcd\" (UniqueName: \"kubernetes.io/projected/615d0917-5b05-44ad-856a-f7e723ba954d-kube-api-access-knvcd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.549260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.549166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.649763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.649726    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.649763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.649762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knvcd\" (UniqueName: \"kubernetes.io/projected/615d0917-5b05-44ad-856a-f7e723ba954d-kube-api-access-knvcd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.649786    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.649812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/615d0917-5b05-44ad-856a-f7e723ba954d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.649839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/615d0917-5b05-44ad-856a-f7e723ba954d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.649859    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.649885    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.649905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/615d0917-5b05-44ad-856a-f7e723ba954d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.650258    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.650293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.650370    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.650410    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650747 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.650611    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.650783 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.650767    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/615d0917-5b05-44ad-856a-f7e723ba954d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.652414 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.652397    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/615d0917-5b05-44ad-856a-f7e723ba954d-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.652856 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.652836    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/615d0917-5b05-44ad-856a-f7e723ba954d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.658529 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.658508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knvcd\" (UniqueName: \"kubernetes.io/projected/615d0917-5b05-44ad-856a-f7e723ba954d-kube-api-access-knvcd\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.658623 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.658510    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/615d0917-5b05-44ad-856a-f7e723ba954d-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-k5pcr\" (UID: \"615d0917-5b05-44ad-856a-f7e723ba954d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:39.757850 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:39.757764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:40.034152 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:40.034114    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"]
Sep 17 23:13:40.035643 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:13:40.035613    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615d0917_5b05_44ad_856a_f7e723ba954d.slice/crio-20cad49812bfaf6e817a0c550299d5778201bfb126dcbcddcb450652b753cc75 WatchSource:0}: Error finding container 20cad49812bfaf6e817a0c550299d5778201bfb126dcbcddcb450652b753cc75: Status 404 returned error can't find the container with id 20cad49812bfaf6e817a0c550299d5778201bfb126dcbcddcb450652b753cc75
Sep 17 23:13:40.037782 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:40.037749    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:13:40.037888 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:40.037838    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:13:40.037888 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:40.037881    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:13:40.073431 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:40.073404    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" event={"ID":"615d0917-5b05-44ad-856a-f7e723ba954d","Type":"ContainerStarted","Data":"20cad49812bfaf6e817a0c550299d5778201bfb126dcbcddcb450652b753cc75"}
Sep 17 23:13:41.082334 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:41.082286    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" event={"ID":"615d0917-5b05-44ad-856a-f7e723ba954d","Type":"ContainerStarted","Data":"7d015fd2ee1a7c16d48d529606a856dea672ad093150ef61bd65aab8dcc18d31"}
Sep 17 23:13:41.103088 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:41.102962    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" podStartSLOduration=2.102943974 podStartE2EDuration="2.102943974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:13:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:13:41.102221946 +0000 UTC m=+441.106413786" watchObservedRunningTime="2026-09-17 23:13:41.102943974 +0000 UTC m=+441.107135813"
Sep 17 23:13:41.758116 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:41.758081    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:41.762678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:41.762652    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:42.086517 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:42.086442    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:42.087472 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:42.087457    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:13:45.849019 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:45.848983    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-bt9mh"]
Sep 17 23:13:45.852434 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:45.852417    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-bt9mh"
Sep 17 23:13:45.857245 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:45.857223    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 17 23:13:45.857373 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:45.857301    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 17 23:13:45.858250 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:45.858233    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cgpp7\""
Sep 17 23:13:45.864722 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:45.864701    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bt9mh"]
Sep 17 23:13:45.904423 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:45.904399    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvql6\" (UniqueName: \"kubernetes.io/projected/963dcecd-9cb9-4a3a-8629-b3005e762880-kube-api-access-bvql6\") pod \"authorino-59c974bd48-bt9mh\" (UID: \"963dcecd-9cb9-4a3a-8629-b3005e762880\") " pod="kuadrant-system/authorino-59c974bd48-bt9mh"
Sep 17 23:13:46.009872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:46.009836    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvql6\" (UniqueName: \"kubernetes.io/projected/963dcecd-9cb9-4a3a-8629-b3005e762880-kube-api-access-bvql6\") pod \"authorino-59c974bd48-bt9mh\" (UID: \"963dcecd-9cb9-4a3a-8629-b3005e762880\") " pod="kuadrant-system/authorino-59c974bd48-bt9mh"
Sep 17 23:13:46.018890 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:46.018866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvql6\" (UniqueName: \"kubernetes.io/projected/963dcecd-9cb9-4a3a-8629-b3005e762880-kube-api-access-bvql6\") pod \"authorino-59c974bd48-bt9mh\" (UID: \"963dcecd-9cb9-4a3a-8629-b3005e762880\") " pod="kuadrant-system/authorino-59c974bd48-bt9mh"
Sep 17 23:13:46.161142 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:46.161084    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-bt9mh"
Sep 17 23:13:46.428171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:46.428147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bt9mh"]
Sep 17 23:13:46.430245 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:13:46.430221    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod963dcecd_9cb9_4a3a_8629_b3005e762880.slice/crio-ab6c7218d19c8c2084de027e1aa3cc073987f56cb521d2f7f69ec1c5a32ab47a WatchSource:0}: Error finding container ab6c7218d19c8c2084de027e1aa3cc073987f56cb521d2f7f69ec1c5a32ab47a: Status 404 returned error can't find the container with id ab6c7218d19c8c2084de027e1aa3cc073987f56cb521d2f7f69ec1c5a32ab47a
Sep 17 23:13:47.111873 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:47.111839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-bt9mh" event={"ID":"963dcecd-9cb9-4a3a-8629-b3005e762880","Type":"ContainerStarted","Data":"ab6c7218d19c8c2084de027e1aa3cc073987f56cb521d2f7f69ec1c5a32ab47a"}
Sep 17 23:13:50.130460 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:50.130423    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-bt9mh" event={"ID":"963dcecd-9cb9-4a3a-8629-b3005e762880","Type":"ContainerStarted","Data":"d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29"}
Sep 17 23:13:50.146674 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:13:50.146627    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-bt9mh" podStartSLOduration=2.344155586 podStartE2EDuration="5.146610946s" totalImagesPullingTime="2.80245536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:13:50.144727786 +0000 UTC m=+450.148919626" watchObservedRunningTime="2026-09-17 23:13:50.146610946 +0000 UTC m=+450.150802827"
Sep 17 23:14:17.347979 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:17.347944    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bt9mh"]
Sep 17 23:14:17.348533 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:17.348116    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-bt9mh" podUID="963dcecd-9cb9-4a3a-8629-b3005e762880" containerName="authorino" containerID="cri-o://d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29" gracePeriod=30
Sep 17 23:14:17.599309 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:17.599252    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-bt9mh"
Sep 17 23:14:17.678644 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:17.678609    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvql6\" (UniqueName: \"kubernetes.io/projected/963dcecd-9cb9-4a3a-8629-b3005e762880-kube-api-access-bvql6\") pod \"963dcecd-9cb9-4a3a-8629-b3005e762880\" (UID: \"963dcecd-9cb9-4a3a-8629-b3005e762880\") "
Sep 17 23:14:17.680756 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:17.680731    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/963dcecd-9cb9-4a3a-8629-b3005e762880-kube-api-access-bvql6" (OuterVolumeSpecName: "kube-api-access-bvql6") pod "963dcecd-9cb9-4a3a-8629-b3005e762880" (UID: "963dcecd-9cb9-4a3a-8629-b3005e762880"). InnerVolumeSpecName "kube-api-access-bvql6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:14:17.779329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:17.779297    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvql6\" (UniqueName: \"kubernetes.io/projected/963dcecd-9cb9-4a3a-8629-b3005e762880-kube-api-access-bvql6\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:14:18.274690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.274602    2581 generic.go:356] "Generic (PLEG): container finished" podID="963dcecd-9cb9-4a3a-8629-b3005e762880" containerID="d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29" exitCode=0
Sep 17 23:14:18.274690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.274639    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-bt9mh"
Sep 17 23:14:18.274690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.274679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-bt9mh" event={"ID":"963dcecd-9cb9-4a3a-8629-b3005e762880","Type":"ContainerDied","Data":"d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29"}
Sep 17 23:14:18.274916 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.274709    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-bt9mh" event={"ID":"963dcecd-9cb9-4a3a-8629-b3005e762880","Type":"ContainerDied","Data":"ab6c7218d19c8c2084de027e1aa3cc073987f56cb521d2f7f69ec1c5a32ab47a"}
Sep 17 23:14:18.274916 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.274729    2581 scope.go:117] "RemoveContainer" containerID="d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29"
Sep 17 23:14:18.283774 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.283757    2581 scope.go:117] "RemoveContainer" containerID="d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29"
Sep 17 23:14:18.284044 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:14:18.284027    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29\": container with ID starting with d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29 not found: ID does not exist" containerID="d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29"
Sep 17 23:14:18.284090 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.284053    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29"} err="failed to get container status \"d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29\": rpc error: code = NotFound desc = could not find container \"d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29\": container with ID starting with d6dcf194e23ee9402c6e6a8909e3e68e841fc997cd70c31f28bc19a1aa14be29 not found: ID does not exist"
Sep 17 23:14:18.290820 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.290799    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bt9mh"]
Sep 17 23:14:18.294699 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:18.294673    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-bt9mh"]
Sep 17 23:14:19.997259 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:19.997213    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="963dcecd-9cb9-4a3a-8629-b3005e762880" path="/var/lib/kubelet/pods/963dcecd-9cb9-4a3a-8629-b3005e762880/volumes"
Sep 17 23:14:22.284430 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.284351    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76fc7c8464-p68dq"]
Sep 17 23:14:22.284796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.284673    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="963dcecd-9cb9-4a3a-8629-b3005e762880" containerName="authorino"
Sep 17 23:14:22.284796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.284684    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="963dcecd-9cb9-4a3a-8629-b3005e762880" containerName="authorino"
Sep 17 23:14:22.284796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.284749    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="963dcecd-9cb9-4a3a-8629-b3005e762880" containerName="authorino"
Sep 17 23:14:22.290446 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.290423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.293013 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.292987    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 17 23:14:22.293158 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.293044    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 17 23:14:22.293158 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.293102    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 17 23:14:22.293351 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.293333    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-snfm4\""
Sep 17 23:14:22.298867 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.298847    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76fc7c8464-p68dq"]
Sep 17 23:14:22.316884 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.316861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-metrics-certs\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.317021 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.316904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.317021 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.316985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r9j2\" (UniqueName: \"kubernetes.io/projected/dd053881-ce5c-448e-becf-0a7e7f4499c7-kube-api-access-4r9j2\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.418356 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.418322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.418530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.418381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r9j2\" (UniqueName: \"kubernetes.io/projected/dd053881-ce5c-448e-becf-0a7e7f4499c7-kube-api-access-4r9j2\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.418530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.418412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-metrics-certs\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.418530 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:14:22.418472    2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 17 23:14:22.418652 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:14:22.418540    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert podName:dd053881-ce5c-448e-becf-0a7e7f4499c7 nodeName:}" failed. No retries permitted until 2026-09-17 23:14:22.918524384 +0000 UTC m=+482.922716205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert") pod "maas-controller-76fc7c8464-p68dq" (UID: "dd053881-ce5c-448e-becf-0a7e7f4499c7") : secret "maas-controller-webhook-cert" not found
Sep 17 23:14:22.420837 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.420815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-metrics-certs\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.427515 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.427495    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r9j2\" (UniqueName: \"kubernetes.io/projected/dd053881-ce5c-448e-becf-0a7e7f4499c7-kube-api-access-4r9j2\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.464061 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.464035    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-565f9646f6-6p9fd"]
Sep 17 23:14:22.466687 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.466673    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.478298 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.478273    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-565f9646f6-6p9fd"]
Sep 17 23:14:22.519027 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.518996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-webhook-cert\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.519201 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.519043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r7jm\" (UniqueName: \"kubernetes.io/projected/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-kube-api-access-8r7jm\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.519201 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.519092    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-metrics-certs\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.619853 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.619824    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-webhook-cert\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.620012 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.619874    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r7jm\" (UniqueName: \"kubernetes.io/projected/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-kube-api-access-8r7jm\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.620012 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.619905    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-metrics-certs\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.622307 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.622283    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-metrics-certs\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.622428 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.622307    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-webhook-cert\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.627461 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.627437    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r7jm\" (UniqueName: \"kubernetes.io/projected/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-kube-api-access-8r7jm\") pod \"maas-controller-565f9646f6-6p9fd\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") " pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.776878 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.776844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:22.922290 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.922209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:22.924527 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:22.924508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert\") pod \"maas-controller-76fc7c8464-p68dq\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") " pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:23.057676 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:23.057638    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-565f9646f6-6p9fd"]
Sep 17 23:14:23.062998 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:14:23.062959    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c06dbcd_7fcd_4c0b_9e7b_9291a09ea72b.slice/crio-0c9cc6cb8bb73435607ea8ea0127acd90a2bbbd8df0a58891f666c10c53c2b1f WatchSource:0}: Error finding container 0c9cc6cb8bb73435607ea8ea0127acd90a2bbbd8df0a58891f666c10c53c2b1f: Status 404 returned error can't find the container with id 0c9cc6cb8bb73435607ea8ea0127acd90a2bbbd8df0a58891f666c10c53c2b1f
Sep 17 23:14:23.202104 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:23.202015    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:23.303012 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:23.302981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-6p9fd" event={"ID":"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b","Type":"ContainerStarted","Data":"0c9cc6cb8bb73435607ea8ea0127acd90a2bbbd8df0a58891f666c10c53c2b1f"}
Sep 17 23:14:23.474605 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:23.474582    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76fc7c8464-p68dq"]
Sep 17 23:14:23.476159 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:14:23.476114    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd053881_ce5c_448e_becf_0a7e7f4499c7.slice/crio-ca0900f7847152e3937de39225df0e74998a39c6023dec042b4bdf054dfda955 WatchSource:0}: Error finding container ca0900f7847152e3937de39225df0e74998a39c6023dec042b4bdf054dfda955: Status 404 returned error can't find the container with id ca0900f7847152e3937de39225df0e74998a39c6023dec042b4bdf054dfda955
Sep 17 23:14:24.311047 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:24.311004    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76fc7c8464-p68dq" event={"ID":"dd053881-ce5c-448e-becf-0a7e7f4499c7","Type":"ContainerStarted","Data":"ca0900f7847152e3937de39225df0e74998a39c6023dec042b4bdf054dfda955"}
Sep 17 23:14:26.324848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:26.324809    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-6p9fd" event={"ID":"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b","Type":"ContainerStarted","Data":"a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1"}
Sep 17 23:14:26.325290 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:26.324876    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:26.326292 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:26.326271    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76fc7c8464-p68dq" event={"ID":"dd053881-ce5c-448e-becf-0a7e7f4499c7","Type":"ContainerStarted","Data":"f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1"}
Sep 17 23:14:26.326397 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:26.326384    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:26.351008 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:26.350961    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-565f9646f6-6p9fd" podStartSLOduration=1.8365331729999999 podStartE2EDuration="4.350947332s" totalImagesPullingTime="2.514414159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:14:26.349011018 +0000 UTC m=+486.353202880" watchObservedRunningTime="2026-09-17 23:14:26.350947332 +0000 UTC m=+486.355139171"
Sep 17 23:14:26.394671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:26.394615    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-76fc7c8464-p68dq" podStartSLOduration=2.283211756 podStartE2EDuration="4.394598525s" totalImagesPullingTime="2.111386769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:14:26.393257732 +0000 UTC m=+486.397449573" watchObservedRunningTime="2026-09-17 23:14:26.394598525 +0000 UTC m=+486.398790367"
Sep 17 23:14:37.333942 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.333912    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:14:37.334327 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.333984    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:37.402648 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.402611    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76fc7c8464-p68dq"]
Sep 17 23:14:37.402832 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.402799    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-76fc7c8464-p68dq" podUID="dd053881-ce5c-448e-becf-0a7e7f4499c7" containerName="manager" containerID="cri-o://f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1" gracePeriod=30
Sep 17 23:14:37.640596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.640573    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:37.753227 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.753197    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert\") pod \"dd053881-ce5c-448e-becf-0a7e7f4499c7\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") "
Sep 17 23:14:37.753392 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.753243    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r9j2\" (UniqueName: \"kubernetes.io/projected/dd053881-ce5c-448e-becf-0a7e7f4499c7-kube-api-access-4r9j2\") pod \"dd053881-ce5c-448e-becf-0a7e7f4499c7\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") "
Sep 17 23:14:37.753392 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.753268    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-metrics-certs\") pod \"dd053881-ce5c-448e-becf-0a7e7f4499c7\" (UID: \"dd053881-ce5c-448e-becf-0a7e7f4499c7\") "
Sep 17 23:14:37.755439 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.755404    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "dd053881-ce5c-448e-becf-0a7e7f4499c7" (UID: "dd053881-ce5c-448e-becf-0a7e7f4499c7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:14:37.755572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.755449    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dd053881-ce5c-448e-becf-0a7e7f4499c7" (UID: "dd053881-ce5c-448e-becf-0a7e7f4499c7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:14:37.755572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.755467    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd053881-ce5c-448e-becf-0a7e7f4499c7-kube-api-access-4r9j2" (OuterVolumeSpecName: "kube-api-access-4r9j2") pod "dd053881-ce5c-448e-becf-0a7e7f4499c7" (UID: "dd053881-ce5c-448e-becf-0a7e7f4499c7"). InnerVolumeSpecName "kube-api-access-4r9j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:14:37.854502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.854467    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-webhook-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:14:37.854502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.854498    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r9j2\" (UniqueName: \"kubernetes.io/projected/dd053881-ce5c-448e-becf-0a7e7f4499c7-kube-api-access-4r9j2\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:14:37.854502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:37.854508    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd053881-ce5c-448e-becf-0a7e7f4499c7-metrics-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:14:38.388500 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.388459    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd053881-ce5c-448e-becf-0a7e7f4499c7" containerID="f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1" exitCode=0
Sep 17 23:14:38.388936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.388507    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76fc7c8464-p68dq"
Sep 17 23:14:38.388936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.388517    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76fc7c8464-p68dq" event={"ID":"dd053881-ce5c-448e-becf-0a7e7f4499c7","Type":"ContainerDied","Data":"f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1"}
Sep 17 23:14:38.388936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.388568    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76fc7c8464-p68dq" event={"ID":"dd053881-ce5c-448e-becf-0a7e7f4499c7","Type":"ContainerDied","Data":"ca0900f7847152e3937de39225df0e74998a39c6023dec042b4bdf054dfda955"}
Sep 17 23:14:38.388936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.388599    2581 scope.go:117] "RemoveContainer" containerID="f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1"
Sep 17 23:14:38.396993 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.396979    2581 scope.go:117] "RemoveContainer" containerID="f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1"
Sep 17 23:14:38.397261 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:14:38.397244    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1\": container with ID starting with f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1 not found: ID does not exist" containerID="f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1"
Sep 17 23:14:38.397315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.397269    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1"} err="failed to get container status \"f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1\": rpc error: code = NotFound desc = could not find container \"f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1\": container with ID starting with f3c5dec5a04e7241aa92a95bd963f883221912f6680e39b2b50610ca0084fda1 not found: ID does not exist"
Sep 17 23:14:38.405556 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.405518    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76fc7c8464-p68dq"]
Sep 17 23:14:38.407882 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:38.407861    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76fc7c8464-p68dq"]
Sep 17 23:14:39.992686 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:39.992656    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd053881-ce5c-448e-becf-0a7e7f4499c7" path="/var/lib/kubelet/pods/dd053881-ce5c-448e-becf-0a7e7f4499c7/volumes"
Sep 17 23:14:56.152521 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.152481    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"]
Sep 17 23:14:56.152983 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.152811    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd053881-ce5c-448e-becf-0a7e7f4499c7" containerName="manager"
Sep 17 23:14:56.152983 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.152822    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd053881-ce5c-448e-becf-0a7e7f4499c7" containerName="manager"
Sep 17 23:14:56.152983 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.152881    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd053881-ce5c-448e-becf-0a7e7f4499c7" containerName="manager"
Sep 17 23:14:56.157083 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.157067    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:56.159678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.159657    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lmktb\""
Sep 17 23:14:56.159882 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.159870    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 17 23:14:56.165711 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.165684    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"]
Sep 17 23:14:56.314261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.314217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9babfa47-30fb-4a73-88f9-8a090d496b96-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-kd6gq\" (UID: \"9babfa47-30fb-4a73-88f9-8a090d496b96\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:56.314261 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.314260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-692vn\" (UniqueName: \"kubernetes.io/projected/9babfa47-30fb-4a73-88f9-8a090d496b96-kube-api-access-692vn\") pod \"payload-processing-5b54b7fbb7-kd6gq\" (UID: \"9babfa47-30fb-4a73-88f9-8a090d496b96\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:56.415917 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.415821    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9babfa47-30fb-4a73-88f9-8a090d496b96-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-kd6gq\" (UID: \"9babfa47-30fb-4a73-88f9-8a090d496b96\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:56.415917 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.415883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-692vn\" (UniqueName: \"kubernetes.io/projected/9babfa47-30fb-4a73-88f9-8a090d496b96-kube-api-access-692vn\") pod \"payload-processing-5b54b7fbb7-kd6gq\" (UID: \"9babfa47-30fb-4a73-88f9-8a090d496b96\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:56.416646 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.416621    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9babfa47-30fb-4a73-88f9-8a090d496b96-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-kd6gq\" (UID: \"9babfa47-30fb-4a73-88f9-8a090d496b96\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:56.424737 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.424707    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-692vn\" (UniqueName: \"kubernetes.io/projected/9babfa47-30fb-4a73-88f9-8a090d496b96-kube-api-access-692vn\") pod \"payload-processing-5b54b7fbb7-kd6gq\" (UID: \"9babfa47-30fb-4a73-88f9-8a090d496b96\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:56.468354 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.468322    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:56.748527 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:56.748442    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"]
Sep 17 23:14:56.751477 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:14:56.751449    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9babfa47_30fb_4a73_88f9_8a090d496b96.slice/crio-d725e279ca750acd53030859ac3f7f2475fb1c7a789c5ecc15d84baf1f236102 WatchSource:0}: Error finding container d725e279ca750acd53030859ac3f7f2475fb1c7a789c5ecc15d84baf1f236102: Status 404 returned error can't find the container with id d725e279ca750acd53030859ac3f7f2475fb1c7a789c5ecc15d84baf1f236102
Sep 17 23:14:57.487575 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:57.487539    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq" event={"ID":"9babfa47-30fb-4a73-88f9-8a090d496b96","Type":"ContainerStarted","Data":"d725e279ca750acd53030859ac3f7f2475fb1c7a789c5ecc15d84baf1f236102"}
Sep 17 23:14:58.495046 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:58.494961    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq" event={"ID":"9babfa47-30fb-4a73-88f9-8a090d496b96","Type":"ContainerStarted","Data":"8a209cc499d6a0f60c4471fe412d1e6b70a3b3af1e73f10f8cf966e250fec370"}
Sep 17 23:14:58.495046 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:58.495025    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:14:58.514075 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:14:58.514032    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq" podStartSLOduration=1.071400991 podStartE2EDuration="2.514018267s" totalImagesPullingTime="1.442617276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:14:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:14:58.511792014 +0000 UTC m=+518.515983854" watchObservedRunningTime="2026-09-17 23:14:58.514018267 +0000 UTC m=+518.518210106"
Sep 17 23:15:05.532953 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:05.532922    2581 generic.go:356] "Generic (PLEG): container finished" podID="615d0917-5b05-44ad-856a-f7e723ba954d" containerID="7d015fd2ee1a7c16d48d529606a856dea672ad093150ef61bd65aab8dcc18d31" exitCode=0
Sep 17 23:15:05.533434 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:05.532997    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" event={"ID":"615d0917-5b05-44ad-856a-f7e723ba954d","Type":"ContainerDied","Data":"7d015fd2ee1a7c16d48d529606a856dea672ad093150ef61bd65aab8dcc18d31"}
Sep 17 23:15:05.533505 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:05.533479    2581 scope.go:117] "RemoveContainer" containerID="7d015fd2ee1a7c16d48d529606a856dea672ad093150ef61bd65aab8dcc18d31"
Sep 17 23:15:05.534486 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:05.534459    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:15:05.534580 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:05.534535    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:15:05.534645 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:05.534597    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:15:06.540000 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:06.539965    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" event={"ID":"615d0917-5b05-44ad-856a-f7e723ba954d","Type":"ContainerStarted","Data":"2fcdfe8081411e32a31f3f4a3f7fdb841dbc7baf89a5dad3b59033e043dd8bac"}
Sep 17 23:15:06.758453 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:06.758363    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:15:06.759048 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:06.758673    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-k5pcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 17 23:15:06.759048 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:06.758742    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" podUID="615d0917-5b05-44ad-856a-f7e723ba954d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 17 23:15:07.759196 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:07.758717    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-k5pcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 17 23:15:07.759196 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:07.758782    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" podUID="615d0917-5b05-44ad-856a-f7e723ba954d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 17 23:15:08.758393 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:08.758356    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-k5pcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 17 23:15:08.758686 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:08.758657    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" podUID="615d0917-5b05-44ad-856a-f7e723ba954d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 17 23:15:09.502584 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:09.502556    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-kd6gq"
Sep 17 23:15:09.759012 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:09.758909    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-k5pcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 17 23:15:09.759012 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:09.758980    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" podUID="615d0917-5b05-44ad-856a-f7e723ba954d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 17 23:15:10.758391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:10.758352    2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-k5pcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Sep 17 23:15:10.758973 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:10.758936    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr" podUID="615d0917-5b05-44ad-856a-f7e723ba954d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Sep 17 23:15:11.762570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:11.762534    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:15:11.763039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:11.762913    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:15:11.763650 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:11.763631    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-k5pcr"
Sep 17 23:15:34.545049 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.545010    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"]
Sep 17 23:15:34.551867 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.551846    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.554855 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.554711    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 17 23:15:34.556342 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.555984    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 17 23:15:34.556342 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.556044    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-frvp9\""
Sep 17 23:15:34.556342 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.555984    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 17 23:15:34.558911 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.558888    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"]
Sep 17 23:15:34.670787 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.670756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6p7n\" (UniqueName: \"kubernetes.io/projected/b051d47e-06a6-47fe-9ffd-83dcffbec306-kube-api-access-d6p7n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.670787 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.670797    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.671024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.670828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.671024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.670932    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.671024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.670973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b051d47e-06a6-47fe-9ffd-83dcffbec306-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.671024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.670996    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772008 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.771974    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.772034    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6p7n\" (UniqueName: \"kubernetes.io/projected/b051d47e-06a6-47fe-9ffd-83dcffbec306-kube-api-access-d6p7n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.772058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.772074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.772151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.772194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b051d47e-06a6-47fe-9ffd-83dcffbec306-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772593 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.772559    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772593 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.772582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.772727 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.772615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.774302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.774280    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b051d47e-06a6-47fe-9ffd-83dcffbec306-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.774554 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.774539    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b051d47e-06a6-47fe-9ffd-83dcffbec306-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.779926 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.779905    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6p7n\" (UniqueName: \"kubernetes.io/projected/b051d47e-06a6-47fe-9ffd-83dcffbec306-kube-api-access-d6p7n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jzh59\" (UID: \"b051d47e-06a6-47fe-9ffd-83dcffbec306\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:34.863964 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:34.863932    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:35.138750 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:35.138723    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"]
Sep 17 23:15:35.140393 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:15:35.140368    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb051d47e_06a6_47fe_9ffd_83dcffbec306.slice/crio-e05456d87441509df09ea91a61ced283479629f21d55aa677e25867a352b162a WatchSource:0}: Error finding container e05456d87441509df09ea91a61ced283479629f21d55aa677e25867a352b162a: Status 404 returned error can't find the container with id e05456d87441509df09ea91a61ced283479629f21d55aa677e25867a352b162a
Sep 17 23:15:35.700046 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:35.700011    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59" event={"ID":"b051d47e-06a6-47fe-9ffd-83dcffbec306","Type":"ContainerStarted","Data":"e05456d87441509df09ea91a61ced283479629f21d55aa677e25867a352b162a"}
Sep 17 23:15:36.706303 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:36.706262    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59" event={"ID":"b051d47e-06a6-47fe-9ffd-83dcffbec306","Type":"ContainerStarted","Data":"f53062c63483c47be3cc330517e6aab23f22c55a2ca9e53bb57db1c13a37001a"}
Sep 17 23:15:36.725460 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:36.725411    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59" podStartSLOduration=1.721154063 podStartE2EDuration="2.725395654s" totalImagesPullingTime="1.004241591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:15:36.722901419 +0000 UTC m=+556.727093272" watchObservedRunningTime="2026-09-17 23:15:36.725395654 +0000 UTC m=+556.729587494"
Sep 17 23:15:44.864196 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:44.864159    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:44.864780 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:44.864329    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:44.876943 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:44.876920    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:15:45.764534 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:15:45.764496    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jzh59"
Sep 17 23:16:03.935981 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:03.935941    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"]
Sep 17 23:16:03.940813 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:03.940792    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:03.943677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:03.943654    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 17 23:16:03.950085 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:03.950060    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"]
Sep 17 23:16:04.029763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.029725    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.029763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.029757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.030039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.029778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/832b8fb5-86ff-4055-8537-a69fac42b24a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.030039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.029882    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mlt4\" (UniqueName: \"kubernetes.io/projected/832b8fb5-86ff-4055-8537-a69fac42b24a-kube-api-access-4mlt4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.030039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.029927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.030039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.030004    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.130755 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.130716    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.130925 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.130792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.130925 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.130854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.130925 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.130881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.130925 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.130902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/832b8fb5-86ff-4055-8537-a69fac42b24a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.131114 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.130946    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mlt4\" (UniqueName: \"kubernetes.io/projected/832b8fb5-86ff-4055-8537-a69fac42b24a-kube-api-access-4mlt4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.131256 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.131235    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.131322 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.131293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.131395 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.131375    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.133139 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.133108    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/832b8fb5-86ff-4055-8537-a69fac42b24a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.133439 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.133418    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/832b8fb5-86ff-4055-8537-a69fac42b24a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.138935 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.138909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mlt4\" (UniqueName: \"kubernetes.io/projected/832b8fb5-86ff-4055-8537-a69fac42b24a-kube-api-access-4mlt4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9\" (UID: \"832b8fb5-86ff-4055-8537-a69fac42b24a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.252750 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.252661    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:04.530432 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.530401    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"]
Sep 17 23:16:04.531578 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:16:04.531555    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod832b8fb5_86ff_4055_8537_a69fac42b24a.slice/crio-0fa0d46273a2328f624b6900f15d924cdf559480e89adabdce45530d3a9c4e6a WatchSource:0}: Error finding container 0fa0d46273a2328f624b6900f15d924cdf559480e89adabdce45530d3a9c4e6a: Status 404 returned error can't find the container with id 0fa0d46273a2328f624b6900f15d924cdf559480e89adabdce45530d3a9c4e6a
Sep 17 23:16:04.852507 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.852420    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9" event={"ID":"832b8fb5-86ff-4055-8537-a69fac42b24a","Type":"ContainerStarted","Data":"9d028c9c4fa0c0a38ed2674de5dcc1ee429c8efbe2c6ae211ea23f7c117a0bd2"}
Sep 17 23:16:04.852507 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.852477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9" event={"ID":"832b8fb5-86ff-4055-8537-a69fac42b24a","Type":"ContainerStarted","Data":"0fa0d46273a2328f624b6900f15d924cdf559480e89adabdce45530d3a9c4e6a"}
Sep 17 23:16:04.873923 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:04.873869    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9" podStartSLOduration=1.699225487 podStartE2EDuration="1.873853944s" totalImagesPullingTime="174.628457ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:16:04.87223177 +0000 UTC m=+584.876423610" watchObservedRunningTime="2026-09-17 23:16:04.873853944 +0000 UTC m=+584.878045794"
Sep 17 23:16:14.253793 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:14.253756    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:14.254230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:14.253809    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:14.266296 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:14.266272    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:16:14.913527 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:16:14.913497    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9"
Sep 17 23:17:20.537038 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.537000    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b4c854f84-t9xxj"]
Sep 17 23:17:20.541381 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.541352    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b4c854f84-t9xxj"
Sep 17 23:17:20.545770 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.545742    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 17 23:17:20.545908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.545788    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 17 23:17:20.545908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.545792    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 17 23:17:20.545908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.545792    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cgpp7\""
Sep 17 23:17:20.549073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.549052    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b4c854f84-t9xxj"]
Sep 17 23:17:20.717842 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.717801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lp2l\" (UniqueName: \"kubernetes.io/projected/f9cdf16e-cddb-417d-a457-cc347a236cbd-kube-api-access-4lp2l\") pod \"authorino-6b4c854f84-t9xxj\" (UID: \"f9cdf16e-cddb-417d-a457-cc347a236cbd\") " pod="kuadrant-system/authorino-6b4c854f84-t9xxj"
Sep 17 23:17:20.718017 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.717861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9cdf16e-cddb-417d-a457-cc347a236cbd-tls-cert\") pod \"authorino-6b4c854f84-t9xxj\" (UID: \"f9cdf16e-cddb-417d-a457-cc347a236cbd\") " pod="kuadrant-system/authorino-6b4c854f84-t9xxj"
Sep 17 23:17:20.818867 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.818779    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9cdf16e-cddb-417d-a457-cc347a236cbd-tls-cert\") pod \"authorino-6b4c854f84-t9xxj\" (UID: \"f9cdf16e-cddb-417d-a457-cc347a236cbd\") " pod="kuadrant-system/authorino-6b4c854f84-t9xxj"
Sep 17 23:17:20.818996 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.818872    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lp2l\" (UniqueName: \"kubernetes.io/projected/f9cdf16e-cddb-417d-a457-cc347a236cbd-kube-api-access-4lp2l\") pod \"authorino-6b4c854f84-t9xxj\" (UID: \"f9cdf16e-cddb-417d-a457-cc347a236cbd\") " pod="kuadrant-system/authorino-6b4c854f84-t9xxj"
Sep 17 23:17:20.821330 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.821308    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f9cdf16e-cddb-417d-a457-cc347a236cbd-tls-cert\") pod \"authorino-6b4c854f84-t9xxj\" (UID: \"f9cdf16e-cddb-417d-a457-cc347a236cbd\") " pod="kuadrant-system/authorino-6b4c854f84-t9xxj"
Sep 17 23:17:20.828607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.828582    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lp2l\" (UniqueName: \"kubernetes.io/projected/f9cdf16e-cddb-417d-a457-cc347a236cbd-kube-api-access-4lp2l\") pod \"authorino-6b4c854f84-t9xxj\" (UID: \"f9cdf16e-cddb-417d-a457-cc347a236cbd\") " pod="kuadrant-system/authorino-6b4c854f84-t9xxj"
Sep 17 23:17:20.854637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:20.854613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b4c854f84-t9xxj"
Sep 17 23:17:21.123603 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:21.123564    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b4c854f84-t9xxj"]
Sep 17 23:17:21.124730 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:17:21.124708    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9cdf16e_cddb_417d_a457_cc347a236cbd.slice/crio-a02d31b19ab9ca2552e53f06dfb72da229a5d17bcc8c182925731468f37813d6 WatchSource:0}: Error finding container a02d31b19ab9ca2552e53f06dfb72da229a5d17bcc8c182925731468f37813d6: Status 404 returned error can't find the container with id a02d31b19ab9ca2552e53f06dfb72da229a5d17bcc8c182925731468f37813d6
Sep 17 23:17:21.126035 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:21.126019    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:17:21.245691 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:21.245659    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b4c854f84-t9xxj" event={"ID":"f9cdf16e-cddb-417d-a457-cc347a236cbd","Type":"ContainerStarted","Data":"a02d31b19ab9ca2552e53f06dfb72da229a5d17bcc8c182925731468f37813d6"}
Sep 17 23:17:22.253300 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:22.253208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b4c854f84-t9xxj" event={"ID":"f9cdf16e-cddb-417d-a457-cc347a236cbd","Type":"ContainerStarted","Data":"5d5a97bf5b1ec6389938d015dd0218f4ca41dafaa71c98179021924a79cb61e1"}
Sep 17 23:17:22.271529 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:22.271479    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b4c854f84-t9xxj" podStartSLOduration=1.533584862 podStartE2EDuration="2.271462123s" totalImagesPullingTime="737.877261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:17:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:17:22.27018029 +0000 UTC m=+662.274372129" watchObservedRunningTime="2026-09-17 23:17:22.271462123 +0000 UTC m=+662.275653997"
Sep 17 23:17:39.606979 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.606940    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"]
Sep 17 23:17:39.610605 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.610579    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.613069 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.613045    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7aa654-openshift-default-dockercfg-6f4wq\""
Sep 17 23:17:39.623174 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.623147    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"]
Sep 17 23:17:39.676349 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.676166    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"]
Sep 17 23:17:39.680506 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.680483    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.683084 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.683061    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-7ab633-openshift-default-dockercfg-2q2mp\""
Sep 17 23:17:39.690597 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.690557    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-envoy\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.690830 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.690796    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-certs\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.690953 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.690861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2700b74-04f8-475e-9000-057919d586a3-istiod-ca-cert\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.690953 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.690937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2700b74-04f8-475e-9000-057919d586a3-istio-podinfo\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.691076 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.690979    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-socket\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.691076 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.691003    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-data\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.691076 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.691042    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhtfq\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-kube-api-access-dhtfq\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.691277 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.691086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-istio-token\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.691277 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.691166    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-credential-socket\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.693164 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.693110    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"]
Sep 17 23:17:39.792385 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2700b74-04f8-475e-9000-057919d586a3-istiod-ca-cert\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.792572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792394    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-data\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.792572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2700b74-04f8-475e-9000-057919d586a3-istio-podinfo\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.792572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792527    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-istio-token\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.792572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792565    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-envoy\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.792809 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792597    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhtfq\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-kube-api-access-dhtfq\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.792809 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-certs\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.792809 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792658    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-istio-token\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.792809 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792695    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-envoy\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.792809 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792735    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-credential-socket\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.792809 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792769    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49509f48-becc-4a36-9b74-69b886c9f35e-istio-podinfo\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.792809 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-certs\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.793188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792868    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49509f48-becc-4a36-9b74-69b886c9f35e-istiod-ca-cert\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.793188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792941    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-socket\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.793188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.792976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-data\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.793188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.793018    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-socket\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.793188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.793046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-credential-socket\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.793188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.793071    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb5td\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-kube-api-access-xb5td\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.793495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.793188    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2700b74-04f8-475e-9000-057919d586a3-istiod-ca-cert\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.793495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.793357    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-socket\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.793495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.793367    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-credential-socket\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.793495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.793441    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-data\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.793710 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.793562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-certs\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.795168 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.795148    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-envoy\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.795528 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.795508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2700b74-04f8-475e-9000-057919d586a3-istio-podinfo\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.801653 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.801624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhtfq\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-kube-api-access-dhtfq\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.801840 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.801807    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-istio-token\") pod \"e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") " pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.893680 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.893644    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-data\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.893882 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.893691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-istio-token\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.893882 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.893709    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-envoy\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.893882 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.893810    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-certs\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.893881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49509f48-becc-4a36-9b74-69b886c9f35e-istio-podinfo\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.893955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49509f48-becc-4a36-9b74-69b886c9f35e-istiod-ca-cert\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.893988    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-socket\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.894014    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-credential-socket\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.894038    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb5td\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-kube-api-access-xb5td\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.894224    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-data\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.894259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-certs\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894492 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.894366    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-socket\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894492 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.894398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-credential-socket\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.894998 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.894973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49509f48-becc-4a36-9b74-69b886c9f35e-istiod-ca-cert\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.896232 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.896212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-envoy\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.896317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.896300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49509f48-becc-4a36-9b74-69b886c9f35e-istio-podinfo\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.907862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.907837    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb5td\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-kube-api-access-xb5td\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.910858 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.910841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-istio-token\") pod \"e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") " pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:39.921956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.921931    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:39.999718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:39.999690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:40.210733 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.210708    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"]
Sep 17 23:17:40.212346 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:17:40.212318    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2700b74_04f8_475e_9000_057919d586a3.slice/crio-6c722b11acf6d7329cd1161b52d135295956a9475cbfac275001d37c05b2002b WatchSource:0}: Error finding container 6c722b11acf6d7329cd1161b52d135295956a9475cbfac275001d37c05b2002b: Status 404 returned error can't find the container with id 6c722b11acf6d7329cd1161b52d135295956a9475cbfac275001d37c05b2002b
Sep 17 23:17:40.214550 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.214517    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:40.214635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.214582    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:40.214635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.214616    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:40.280519 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.280496    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"]
Sep 17 23:17:40.281352 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:17:40.281332    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49509f48_becc_4a36_9b74_69b886c9f35e.slice/crio-505c041f22e8715c812e8e0f80fee839f7e68d6030039690a49a1aa3e1c7c484 WatchSource:0}: Error finding container 505c041f22e8715c812e8e0f80fee839f7e68d6030039690a49a1aa3e1c7c484: Status 404 returned error can't find the container with id 505c041f22e8715c812e8e0f80fee839f7e68d6030039690a49a1aa3e1c7c484
Sep 17 23:17:40.283654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.283625    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:40.283722 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.283688    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:40.283722 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.283716    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:40.361470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.361431    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" event={"ID":"f2700b74-04f8-475e-9000-057919d586a3","Type":"ContainerStarted","Data":"04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a"}
Sep 17 23:17:40.361614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.361504    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" event={"ID":"f2700b74-04f8-475e-9000-057919d586a3","Type":"ContainerStarted","Data":"6c722b11acf6d7329cd1161b52d135295956a9475cbfac275001d37c05b2002b"}
Sep 17 23:17:40.363377 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.363344    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" event={"ID":"49509f48-becc-4a36-9b74-69b886c9f35e","Type":"ContainerStarted","Data":"59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3"}
Sep 17 23:17:40.363742 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.363394    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" event={"ID":"49509f48-becc-4a36-9b74-69b886c9f35e","Type":"ContainerStarted","Data":"505c041f22e8715c812e8e0f80fee839f7e68d6030039690a49a1aa3e1c7c484"}
Sep 17 23:17:40.386724 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.386665    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" podStartSLOduration=1.386644988 podStartE2EDuration="1.386644988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:17:40.381772865 +0000 UTC m=+680.385964705" watchObservedRunningTime="2026-09-17 23:17:40.386644988 +0000 UTC m=+680.390836830"
Sep 17 23:17:40.403766 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.403179    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" podStartSLOduration=1.403157991 podStartE2EDuration="1.403157991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:17:40.402189837 +0000 UTC m=+680.406381679" watchObservedRunningTime="2026-09-17 23:17:40.403157991 +0000 UTC m=+680.407349831"
Sep 17 23:17:40.922627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:40.922598    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:41.000785 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:41.000754    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:41.005542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:41.005517    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:41.370328 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:41.370256    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:41.371411 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:41.371384    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:41.927659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:41.927627    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:42.376949 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:42.376915    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:42.377700 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:42.377676    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:17:45.714413 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.714382    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"]
Sep 17 23:17:45.718332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.718311    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:45.720936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.720913    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-7aa654-dockercfg-dlkjd\""
Sep 17 23:17:45.721033 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.720954    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-7aa654\""
Sep 17 23:17:45.732788 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.732763    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"]
Sep 17 23:17:45.865777 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.865737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hms87\" (UniqueName: \"kubernetes.io/projected/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-kube-api-access-hms87\") pod \"payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs\" (UID: \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:45.865950 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.865806    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs\" (UID: \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:45.966652 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.966568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hms87\" (UniqueName: \"kubernetes.io/projected/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-kube-api-access-hms87\") pod \"payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs\" (UID: \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:45.966652 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.966631    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs\" (UID: \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:45.967362 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.967340    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs\" (UID: \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:45.975443 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:45.975421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hms87\" (UniqueName: \"kubernetes.io/projected/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-kube-api-access-hms87\") pod \"payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs\" (UID: \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:46.028479 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:46.028442    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:46.313384 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:46.313356    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"]
Sep 17 23:17:46.401719 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:46.401676    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs" event={"ID":"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf","Type":"ContainerStarted","Data":"1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903"}
Sep 17 23:17:46.402103 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:46.401746    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs" event={"ID":"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf","Type":"ContainerStarted","Data":"15f9539925d9bbfc1549e4e681c82d4a62ce946aa5eb0919ce0bce20f776d03a"}
Sep 17 23:17:46.402103 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:46.401885    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:46.423726 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:46.423674    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs" podStartSLOduration=1.4236596559999999 podStartE2EDuration="1.423659656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:17:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:17:46.421914788 +0000 UTC m=+686.426106626" watchObservedRunningTime="2026-09-17 23:17:46.423659656 +0000 UTC m=+686.427851512"
Sep 17 23:17:47.515144 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.515094    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"]
Sep 17 23:17:47.519018 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.518995    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.521689 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.521670    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-6114ed-openshift-default-dockercfg-hnnkg\""
Sep 17 23:17:47.533996 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.533971    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"]
Sep 17 23:17:47.582383 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582354    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd8809bb-864e-4aa6-b44b-e32f268bc647-istiod-ca-cert\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.582549 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-token\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.582549 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582412    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-podinfo\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.582549 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-data\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.582549 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582529    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-envoy\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.582738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-certs\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.582738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-socket\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.582738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p2b8\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-kube-api-access-7p2b8\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.582738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.582675    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-credential-socket\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.683850 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.683801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd8809bb-864e-4aa6-b44b-e32f268bc647-istiod-ca-cert\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.683850 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.683853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-token\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684107 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.683880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-podinfo\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684107 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.683930    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-data\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684107 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.683979    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-envoy\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684305 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-certs\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684305 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-socket\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684305 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p2b8\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-kube-api-access-7p2b8\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684305 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684259    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-credential-socket\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684514 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684436    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-data\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684514 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684476    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-certs\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684631 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684552    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-socket\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684672    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-credential-socket\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.684745 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.684684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd8809bb-864e-4aa6-b44b-e32f268bc647-istiod-ca-cert\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.686804 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.686772    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-envoy\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.686934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.686918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-podinfo\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.692641 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.692608    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p2b8\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-kube-api-access-7p2b8\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.692773 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.692751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-token\") pod \"e2e-autores-6114ed-openshift-default-788fc9544-bsrf4\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") " pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:47.830436 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:47.830333    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:48.162510 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:48.162450    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"]
Sep 17 23:17:48.179072 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:48.178582    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:48.179072 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:48.178676    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:48.179072 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:48.178733    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:48.417555 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:48.417224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" event={"ID":"dd8809bb-864e-4aa6-b44b-e32f268bc647","Type":"ContainerStarted","Data":"279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08"}
Sep 17 23:17:48.417555 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:48.417302    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" event={"ID":"dd8809bb-864e-4aa6-b44b-e32f268bc647","Type":"ContainerStarted","Data":"8b262bbcb2f988de4c9cbfd059c49e5acb6384cb09ef632b1d0188439b476a54"}
Sep 17 23:17:48.440161 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:48.439352    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" podStartSLOduration=1.439337482 podStartE2EDuration="1.439337482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:17:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:17:48.436738082 +0000 UTC m=+688.440929921" watchObservedRunningTime="2026-09-17 23:17:48.439337482 +0000 UTC m=+688.443529322"
Sep 17 23:17:48.831279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:48.831193    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:49.838941 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:49.838910    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:50.073845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.073795    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"]
Sep 17 23:17:50.078255 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.078229    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:50.081444 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.081422    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-0b4d93\""
Sep 17 23:17:50.085451 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.085422    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"]
Sep 17 23:17:50.221869 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.221813    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4glwz\" (UniqueName: \"kubernetes.io/projected/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-kube-api-access-4glwz\") pod \"payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd\" (UID: \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:50.222071 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.221985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd\" (UID: \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:50.322653 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.322612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd\" (UID: \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:50.322836 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.322711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4glwz\" (UniqueName: \"kubernetes.io/projected/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-kube-api-access-4glwz\") pod \"payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd\" (UID: \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:50.323408 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.323383    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd\" (UID: \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:50.330047 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.330017    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4glwz\" (UniqueName: \"kubernetes.io/projected/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-kube-api-access-4glwz\") pod \"payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd\" (UID: \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:50.391915 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.391883    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:50.429367 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.429333    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:50.430422 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.430398    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:17:50.706181 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:50.706145    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"]
Sep 17 23:17:50.707207 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:17:50.707172    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b4b9fa2_6693_4be0_bc8b_c639d5f3191d.slice/crio-87850c1a45736b6f986d4ae55d5f9c4c2c482858b6e62f919af45fa255136489 WatchSource:0}: Error finding container 87850c1a45736b6f986d4ae55d5f9c4c2c482858b6e62f919af45fa255136489: Status 404 returned error can't find the container with id 87850c1a45736b6f986d4ae55d5f9c4c2c482858b6e62f919af45fa255136489
Sep 17 23:17:51.436857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:51.436811    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd" event={"ID":"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d","Type":"ContainerStarted","Data":"5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801"}
Sep 17 23:17:51.437353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:51.436883    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:17:51.437353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:51.436908    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd" event={"ID":"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d","Type":"ContainerStarted","Data":"87850c1a45736b6f986d4ae55d5f9c4c2c482858b6e62f919af45fa255136489"}
Sep 17 23:17:51.463701 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:51.463632    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd" podStartSLOduration=1.46360781 podStartE2EDuration="1.46360781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:17:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:17:51.462542781 +0000 UTC m=+691.466734620" watchObservedRunningTime="2026-09-17 23:17:51.46360781 +0000 UTC m=+691.467799650"
Sep 17 23:17:55.294425 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.294394    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"]
Sep 17 23:17:55.299146 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.299117    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:55.301723 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.301699    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-6114ed\""
Sep 17 23:17:55.301853 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.301778    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-6114ed-dockercfg-rfqdc\""
Sep 17 23:17:55.310715 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.310688    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"]
Sep 17 23:17:55.385854 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.385659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ee358c-688b-4218-8e59-b5b1d08c09ac-plugins-config-volume\") pod \"payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw\" (UID: \"08ee358c-688b-4218-8e59-b5b1d08c09ac\") " pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:55.385854 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.385717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j625t\" (UniqueName: \"kubernetes.io/projected/08ee358c-688b-4218-8e59-b5b1d08c09ac-kube-api-access-j625t\") pod \"payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw\" (UID: \"08ee358c-688b-4218-8e59-b5b1d08c09ac\") " pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:55.487316 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.487278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ee358c-688b-4218-8e59-b5b1d08c09ac-plugins-config-volume\") pod \"payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw\" (UID: \"08ee358c-688b-4218-8e59-b5b1d08c09ac\") " pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:55.487316 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.487317    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j625t\" (UniqueName: \"kubernetes.io/projected/08ee358c-688b-4218-8e59-b5b1d08c09ac-kube-api-access-j625t\") pod \"payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw\" (UID: \"08ee358c-688b-4218-8e59-b5b1d08c09ac\") " pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:55.488013 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.487987    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ee358c-688b-4218-8e59-b5b1d08c09ac-plugins-config-volume\") pod \"payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw\" (UID: \"08ee358c-688b-4218-8e59-b5b1d08c09ac\") " pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:55.495331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.495304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j625t\" (UniqueName: \"kubernetes.io/projected/08ee358c-688b-4218-8e59-b5b1d08c09ac-kube-api-access-j625t\") pod \"payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw\" (UID: \"08ee358c-688b-4218-8e59-b5b1d08c09ac\") " pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:55.612554 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.612519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:55.897681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:55.897635    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"]
Sep 17 23:17:55.898420 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:17:55.898384    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ee358c_688b_4218_8e59_b5b1d08c09ac.slice/crio-906e1915451cccf91718651dcca772aafe9866f02c84e60af8fe11ecc92c0785 WatchSource:0}: Error finding container 906e1915451cccf91718651dcca772aafe9866f02c84e60af8fe11ecc92c0785: Status 404 returned error can't find the container with id 906e1915451cccf91718651dcca772aafe9866f02c84e60af8fe11ecc92c0785
Sep 17 23:17:56.194088 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.193994    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"]
Sep 17 23:17:56.198191 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.198161    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:56.201424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.201398    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f10a7edb-dockercfg-hmmv7\""
Sep 17 23:17:56.201424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.201417    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f10a7edb\""
Sep 17 23:17:56.213133 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.213094    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"]
Sep 17 23:17:56.295266 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.295227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-plugins-config-volume\") pod \"payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8\" (UID: \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\") " pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:56.295680 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.295442    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqtkq\" (UniqueName: \"kubernetes.io/projected/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-kube-api-access-pqtkq\") pod \"payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8\" (UID: \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\") " pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:56.396854 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.396813    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqtkq\" (UniqueName: \"kubernetes.io/projected/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-kube-api-access-pqtkq\") pod \"payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8\" (UID: \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\") " pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:56.397210 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.397187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-plugins-config-volume\") pod \"payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8\" (UID: \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\") " pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:56.398555 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.398530    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-plugins-config-volume\") pod \"payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8\" (UID: \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\") " pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:56.421156 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.421077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqtkq\" (UniqueName: \"kubernetes.io/projected/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-kube-api-access-pqtkq\") pod \"payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8\" (UID: \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\") " pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:56.491339 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.491247    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw" event={"ID":"08ee358c-688b-4218-8e59-b5b1d08c09ac","Type":"ContainerStarted","Data":"d2f494a32a069ef06c3aa025a512d5ee8f6148caeaea6281d28ab5f5ce579843"}
Sep 17 23:17:56.491339 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.491335    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:17:56.491570 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.491360    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw" event={"ID":"08ee358c-688b-4218-8e59-b5b1d08c09ac","Type":"ContainerStarted","Data":"906e1915451cccf91718651dcca772aafe9866f02c84e60af8fe11ecc92c0785"}
Sep 17 23:17:56.509354 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.509280    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw" podStartSLOduration=1.509258371 podStartE2EDuration="1.509258371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:17:56.507578824 +0000 UTC m=+696.511770663" watchObservedRunningTime="2026-09-17 23:17:56.509258371 +0000 UTC m=+696.513450211"
Sep 17 23:17:56.511470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.511450    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:56.822918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:56.822889    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"]
Sep 17 23:17:56.825305 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:17:56.825264    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6335c45f_3de1_4d50_ba1e_2bc49d3d386d.slice/crio-91d84b63eee88d00885092ea28a5049e657d3207e128dc67f9dd6ef664bb5be7 WatchSource:0}: Error finding container 91d84b63eee88d00885092ea28a5049e657d3207e128dc67f9dd6ef664bb5be7: Status 404 returned error can't find the container with id 91d84b63eee88d00885092ea28a5049e657d3207e128dc67f9dd6ef664bb5be7
Sep 17 23:17:57.017666 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.017618    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:56710->10.133.0.43:15021: read: connection reset by peer" start-of-body=
Sep 17 23:17:57.017851 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.017687    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:56710->10.133.0.43:15021: read: connection reset by peer"
Sep 17 23:17:57.408372 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.408344    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:17:57.499919 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.499883    2581 generic.go:356] "Generic (PLEG): container finished" podID="49509f48-becc-4a36-9b74-69b886c9f35e" containerID="59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3" exitCode=0
Sep 17 23:17:57.500092 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.499992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" event={"ID":"49509f48-becc-4a36-9b74-69b886c9f35e","Type":"ContainerDied","Data":"59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3"}
Sep 17 23:17:57.500734 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.500482    2581 scope.go:117] "RemoveContainer" containerID="59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3"
Sep 17 23:17:57.501972 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.501936    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:57.502075 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.502024    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:57.502075 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.502069    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:17:57.503635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.503524    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8" event={"ID":"6335c45f-3de1-4d50-ba1e-2bc49d3d386d","Type":"ContainerStarted","Data":"8ad7fd1e3e4358b51fcfb77cae0747376c9d9d742990563dfec1589842c467e1"}
Sep 17 23:17:57.503635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.503573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8" event={"ID":"6335c45f-3de1-4d50-ba1e-2bc49d3d386d","Type":"ContainerStarted","Data":"91d84b63eee88d00885092ea28a5049e657d3207e128dc67f9dd6ef664bb5be7"}
Sep 17 23:17:57.504240 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.503984    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:17:57.574146 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:57.573949    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8" podStartSLOduration=1.573925252 podStartE2EDuration="1.573925252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:17:57.569588167 +0000 UTC m=+697.573780029" watchObservedRunningTime="2026-09-17 23:17:57.573925252 +0000 UTC m=+697.578117093"
Sep 17 23:17:58.518004 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:58.517959    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" event={"ID":"49509f48-becc-4a36-9b74-69b886c9f35e","Type":"ContainerStarted","Data":"3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc"}
Sep 17 23:17:59.000988 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:59.000954    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:17:59.001393 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:59.001297    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 17 23:17:59.001582 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:17:59.001557    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 17 23:18:00.001060 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:00.001014    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 17 23:18:00.001563 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:00.001084    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 17 23:18:01.000195 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:01.000160    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 17 23:18:01.000501 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:01.000458    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 17 23:18:02.000101 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.000070    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 17 23:18:02.000486 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.000149    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 17 23:18:02.444266 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.444231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:18:02.768977 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.768314    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"]
Sep 17 23:18:02.787637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.783676    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.798025 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.796762    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"]
Sep 17 23:18:02.798025 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.797339    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-8fc0d3-openshift-default-dockercfg-b7vwd\""
Sep 17 23:18:02.876729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.876680    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-credential-socket\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.876933 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.876747    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-token\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.876933 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.876798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txpw4\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-kube-api-access-txpw4\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.876933 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.876855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-envoy\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.876933 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.876909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-certs\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.877291 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.876952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-socket\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.877291 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.876983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-data\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.877291 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.877019    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e49fee3-bc15-48c1-a33f-062892f5d304-istiod-ca-cert\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.877291 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.877047    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-podinfo\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978325    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-envoy\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-certs\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-socket\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-data\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978516    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e49fee3-bc15-48c1-a33f-062892f5d304-istiod-ca-cert\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-podinfo\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-credential-socket\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978655    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-token\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.979495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.978703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txpw4\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-kube-api-access-txpw4\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.982011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.980540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-socket\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.982011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.980798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-certs\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.982011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.981037    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-credential-socket\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.982011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.981694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-data\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.982625 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.982568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-envoy\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.983871 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.983820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-podinfo\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.984236 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.984201    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e49fee3-bc15-48c1-a33f-062892f5d304-istiod-ca-cert\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.991556 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.991534    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txpw4\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-kube-api-access-txpw4\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:02.994059 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:02.994023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-token\") pod \"e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") " pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:03.000608 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.000435    2581 patch_prober.go:28] interesting pod/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body=
Sep 17 23:18:03.000608 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.000489    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused"
Sep 17 23:18:03.104136 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.104015    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:03.416606 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:18:03.416574    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e49fee3_bc15_48c1_a33f_062892f5d304.slice/crio-a4007b7f0f892f7b979160d5ccb2179e453bea7c8c08618224e50423303b10b7 WatchSource:0}: Error finding container a4007b7f0f892f7b979160d5ccb2179e453bea7c8c08618224e50423303b10b7: Status 404 returned error can't find the container with id a4007b7f0f892f7b979160d5ccb2179e453bea7c8c08618224e50423303b10b7
Sep 17 23:18:03.417847 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.417811    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"]
Sep 17 23:18:03.419074 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.419043    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:03.419202 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.419115    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:03.419271 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.419205    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:03.552716 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.552674    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" event={"ID":"5e49fee3-bc15-48c1-a33f-062892f5d304","Type":"ContainerStarted","Data":"b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726"}
Sep 17 23:18:03.552903 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.552736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" event={"ID":"5e49fee3-bc15-48c1-a33f-062892f5d304","Type":"ContainerStarted","Data":"a4007b7f0f892f7b979160d5ccb2179e453bea7c8c08618224e50423303b10b7"}
Sep 17 23:18:03.577499 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:03.577435    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podStartSLOduration=1.577411493 podStartE2EDuration="1.577411493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:18:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:18:03.574923541 +0000 UTC m=+703.579115382" watchObservedRunningTime="2026-09-17 23:18:03.577411493 +0000 UTC m=+703.581603333"
Sep 17 23:18:04.004386 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:04.004358    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:18:04.004828 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:04.004723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:18:04.005475 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:04.005457    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:18:04.104868 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:04.104839    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:04.106208 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:04.106170    2581 patch_prober.go:28] interesting pod/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 17 23:18:04.106343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:04.106234    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 17 23:18:05.110081 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.110054    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:05.568058 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.568023    2581 generic.go:356] "Generic (PLEG): container finished" podID="f2700b74-04f8-475e-9000-057919d586a3" containerID="04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a" exitCode=0
Sep 17 23:18:05.568272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.568099    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" event={"ID":"f2700b74-04f8-475e-9000-057919d586a3","Type":"ContainerDied","Data":"04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a"}
Sep 17 23:18:05.568677 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.568636    2581 scope.go:117] "RemoveContainer" containerID="04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a"
Sep 17 23:18:05.568821 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.568739    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:05.569607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.569483    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:05.570078 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.570014    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:05.570233 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.570096    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:05.570233 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:05.570173    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:06.577140 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:06.577102    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" event={"ID":"f2700b74-04f8-475e-9000-057919d586a3","Type":"ContainerStarted","Data":"c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225"}
Sep 17 23:18:06.922921 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:06.922877    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:18:06.923186 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:06.923159    2581 patch_prober.go:28] interesting pod/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 17 23:18:06.923270 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:06.923229    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 17 23:18:07.506343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:07.506310    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:18:07.923310 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:07.923249    2581 patch_prober.go:28] interesting pod/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 17 23:18:07.923700 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:07.923332    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 17 23:18:08.922807 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:08.922771    2581 patch_prober.go:28] interesting pod/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 17 23:18:08.922995 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:08.922827    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 17 23:18:09.525791 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:09.525754    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:18:09.922450 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:09.922414    2581 patch_prober.go:28] interesting pod/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 17 23:18:09.922656 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:09.922465    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 17 23:18:10.922711 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:10.922677    2581 patch_prober.go:28] interesting pod/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 17 23:18:10.923104 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:10.922730    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 17 23:18:11.923052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:11.923013    2581 patch_prober.go:28] interesting pod/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Sep 17 23:18:11.923470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:11.923069    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Sep 17 23:18:12.378488 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:12.377586    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:18:12.926854 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:12.926823    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:18:12.928188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:12.928161    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:18:16.666852 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:16.666818    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerID="279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08" exitCode=0
Sep 17 23:18:16.667463 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:16.666864    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" event={"ID":"dd8809bb-864e-4aa6-b44b-e32f268bc647","Type":"ContainerDied","Data":"279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08"}
Sep 17 23:18:16.667463 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:16.667392    2581 scope.go:117] "RemoveContainer" containerID="279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08"
Sep 17 23:18:16.668613 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:16.668579    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:16.668698 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:16.668654    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:16.668698 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:16.668687    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:17.679717 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:17.679667    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" event={"ID":"dd8809bb-864e-4aa6-b44b-e32f268bc647","Type":"ContainerStarted","Data":"20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb"}
Sep 17 23:18:17.831789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:17.831033    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:18:17.831963 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:17.831834    2581 patch_prober.go:28] interesting pod/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 17 23:18:17.831963 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:17.831880    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 17 23:18:18.831143 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:18.831092    2581 patch_prober.go:28] interesting pod/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 17 23:18:18.831626 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:18.831182    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 17 23:18:19.831728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:19.831693    2581 patch_prober.go:28] interesting pod/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 17 23:18:19.832209 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:19.831763    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 17 23:18:20.429731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:20.429685    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:18:20.830939 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:20.830856    2581 patch_prober.go:28] interesting pod/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 17 23:18:20.830939 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:20.830924    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 17 23:18:21.831342 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:21.831302    2581 patch_prober.go:28] interesting pod/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 17 23:18:21.831731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:21.831371    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 17 23:18:22.831326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:22.831285    2581 patch_prober.go:28] interesting pod/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Sep 17 23:18:22.831506 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:22.831352    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Sep 17 23:18:23.835734 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:23.835702    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:18:23.836889 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:23.836871    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:18:24.555815 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.555768    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"]
Sep 17 23:18:24.556141 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.556018    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8" podUID="6335c45f-3de1-4d50-ba1e-2bc49d3d386d" containerName="payload-processing" containerID="cri-o://8ad7fd1e3e4358b51fcfb77cae0747376c9d9d742990563dfec1589842c467e1" gracePeriod=30
Sep 17 23:18:24.734818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.734776    2581 generic.go:356] "Generic (PLEG): container finished" podID="6335c45f-3de1-4d50-ba1e-2bc49d3d386d" containerID="8ad7fd1e3e4358b51fcfb77cae0747376c9d9d742990563dfec1589842c467e1" exitCode=0
Sep 17 23:18:24.735094 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.735030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8" event={"ID":"6335c45f-3de1-4d50-ba1e-2bc49d3d386d","Type":"ContainerDied","Data":"8ad7fd1e3e4358b51fcfb77cae0747376c9d9d742990563dfec1589842c467e1"}
Sep 17 23:18:24.833246 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.833209    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:18:24.966828 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.965541    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-plugins-config-volume\") pod \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\" (UID: \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\") "
Sep 17 23:18:24.966828 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.965623    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqtkq\" (UniqueName: \"kubernetes.io/projected/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-kube-api-access-pqtkq\") pod \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\" (UID: \"6335c45f-3de1-4d50-ba1e-2bc49d3d386d\") "
Sep 17 23:18:24.966828 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.966518    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6335c45f-3de1-4d50-ba1e-2bc49d3d386d" (UID: "6335c45f-3de1-4d50-ba1e-2bc49d3d386d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:18:24.984825 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:24.983288    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-kube-api-access-pqtkq" (OuterVolumeSpecName: "kube-api-access-pqtkq") pod "6335c45f-3de1-4d50-ba1e-2bc49d3d386d" (UID: "6335c45f-3de1-4d50-ba1e-2bc49d3d386d"). InnerVolumeSpecName "kube-api-access-pqtkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:18:25.067783 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:25.067684    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:18:25.067783 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:25.067722    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqtkq\" (UniqueName: \"kubernetes.io/projected/6335c45f-3de1-4d50-ba1e-2bc49d3d386d-kube-api-access-pqtkq\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:18:25.743342 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:25.743314    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"
Sep 17 23:18:25.743515 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:25.743314    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8" event={"ID":"6335c45f-3de1-4d50-ba1e-2bc49d3d386d","Type":"ContainerDied","Data":"91d84b63eee88d00885092ea28a5049e657d3207e128dc67f9dd6ef664bb5be7"}
Sep 17 23:18:25.743515 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:25.743444    2581 scope.go:117] "RemoveContainer" containerID="8ad7fd1e3e4358b51fcfb77cae0747376c9d9d742990563dfec1589842c467e1"
Sep 17 23:18:25.766166 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:25.766109    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"]
Sep 17 23:18:25.771306 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:25.771281    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-f10a7edb-89859c5c5-vntv8"]
Sep 17 23:18:26.001901 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:26.001780    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6335c45f-3de1-4d50-ba1e-2bc49d3d386d" path="/var/lib/kubelet/pods/6335c45f-3de1-4d50-ba1e-2bc49d3d386d/volumes"
Sep 17 23:18:26.751877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:26.751843    2581 generic.go:356] "Generic (PLEG): container finished" podID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerID="b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726" exitCode=0
Sep 17 23:18:26.752079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:26.751891    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" event={"ID":"5e49fee3-bc15-48c1-a33f-062892f5d304","Type":"ContainerDied","Data":"b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726"}
Sep 17 23:18:26.752340 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:26.752324    2581 scope.go:117] "RemoveContainer" containerID="b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726"
Sep 17 23:18:26.753569 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:26.753540    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:26.753637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:26.753609    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:26.753673 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:26.753640    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:18:27.763543 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:27.763496    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" event={"ID":"5e49fee3-bc15-48c1-a33f-062892f5d304","Type":"ContainerStarted","Data":"388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106"}
Sep 17 23:18:28.104826 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:28.104743    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:28.104982 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:28.104946    2581 patch_prober.go:28] interesting pod/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 17 23:18:28.105025 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:28.105001    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 17 23:18:29.104653 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:29.104615    2581 patch_prober.go:28] interesting pod/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 17 23:18:29.105047 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:29.104674    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 17 23:18:30.105211 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:30.105174    2581 patch_prober.go:28] interesting pod/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 17 23:18:30.105699 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:30.105237    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 17 23:18:31.105188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.105152    2581 patch_prober.go:28] interesting pod/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 17 23:18:31.105372 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.105237    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 17 23:18:31.964749 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.964714    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"]
Sep 17 23:18:31.969185 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.965677    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6335c45f-3de1-4d50-ba1e-2bc49d3d386d" containerName="payload-processing"
Sep 17 23:18:31.969459 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.969420    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6335c45f-3de1-4d50-ba1e-2bc49d3d386d" containerName="payload-processing"
Sep 17 23:18:31.969856 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.969826    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6335c45f-3de1-4d50-ba1e-2bc49d3d386d" containerName="payload-processing"
Sep 17 23:18:31.982028 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.981993    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"]
Sep 17 23:18:31.982375 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.982344    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:31.989500 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:31.989445    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b75118e5\""
Sep 17 23:18:32.108149 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.105509    2581 patch_prober.go:28] interesting pod/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 17 23:18:32.108149 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.105573    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 17 23:18:32.143257 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.143219    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h5w6\" (UniqueName: \"kubernetes.io/projected/8dad70ff-d8c1-4734-b09b-c808429cc04f-kube-api-access-9h5w6\") pod \"payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl\" (UID: \"8dad70ff-d8c1-4734-b09b-c808429cc04f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:32.143438 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.143283    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dad70ff-d8c1-4734-b09b-c808429cc04f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl\" (UID: \"8dad70ff-d8c1-4734-b09b-c808429cc04f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:32.244095 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.244007    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h5w6\" (UniqueName: \"kubernetes.io/projected/8dad70ff-d8c1-4734-b09b-c808429cc04f-kube-api-access-9h5w6\") pod \"payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl\" (UID: \"8dad70ff-d8c1-4734-b09b-c808429cc04f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:32.244095 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.244056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dad70ff-d8c1-4734-b09b-c808429cc04f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl\" (UID: \"8dad70ff-d8c1-4734-b09b-c808429cc04f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:32.244785 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.244766    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dad70ff-d8c1-4734-b09b-c808429cc04f-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl\" (UID: \"8dad70ff-d8c1-4734-b09b-c808429cc04f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:32.252320 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.252300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h5w6\" (UniqueName: \"kubernetes.io/projected/8dad70ff-d8c1-4734-b09b-c808429cc04f-kube-api-access-9h5w6\") pod \"payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl\" (UID: \"8dad70ff-d8c1-4734-b09b-c808429cc04f\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:32.295975 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.295935    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:32.579021 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.578987    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"]
Sep 17 23:18:32.580758 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:18:32.580710    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dad70ff_d8c1_4734_b09b_c808429cc04f.slice/crio-c04b1b70ab5b964935ef719d967df6d5157b5997a2857b68bb4129d66c9fbc11 WatchSource:0}: Error finding container c04b1b70ab5b964935ef719d967df6d5157b5997a2857b68bb4129d66c9fbc11: Status 404 returned error can't find the container with id c04b1b70ab5b964935ef719d967df6d5157b5997a2857b68bb4129d66c9fbc11
Sep 17 23:18:32.800655 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.800616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl" event={"ID":"8dad70ff-d8c1-4734-b09b-c808429cc04f","Type":"ContainerStarted","Data":"c7c50b738d0dd49650436cc67cada5e2bdce3ebc1a4a437c2c84958683c2c3a1"}
Sep 17 23:18:32.800804 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.800678    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl" event={"ID":"8dad70ff-d8c1-4734-b09b-c808429cc04f","Type":"ContainerStarted","Data":"c04b1b70ab5b964935ef719d967df6d5157b5997a2857b68bb4129d66c9fbc11"}
Sep 17 23:18:32.800880 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.800861    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:32.820098 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:32.820042    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl" podStartSLOduration=1.820028435 podStartE2EDuration="1.820028435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:18:32.817037901 +0000 UTC m=+732.821229739" watchObservedRunningTime="2026-09-17 23:18:32.820028435 +0000 UTC m=+732.824220275"
Sep 17 23:18:33.104743 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:33.104711    2581 patch_prober.go:28] interesting pod/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Sep 17 23:18:33.104984 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:33.104960    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Sep 17 23:18:34.109159 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:34.109107    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:34.109579 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:34.109556    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:34.110316 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:34.110301    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:18:43.809099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:43.809068    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:18:53.956111 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:53.956070    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"]
Sep 17 23:18:53.959744 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:53.959718    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"
Sep 17 23:18:53.962230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:53.962211    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:18:53.962371 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:53.962212    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:18:53.967695 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:53.967666    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"]
Sep 17 23:18:54.064924 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:54.064889    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f47debb-3e28-49fd-944e-b41f08b5903a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8\" (UID: \"5f47debb-3e28-49fd-944e-b41f08b5903a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"
Sep 17 23:18:54.166514 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:54.166477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f47debb-3e28-49fd-944e-b41f08b5903a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8\" (UID: \"5f47debb-3e28-49fd-944e-b41f08b5903a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"
Sep 17 23:18:54.167108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:54.167085    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f47debb-3e28-49fd-944e-b41f08b5903a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8\" (UID: \"5f47debb-3e28-49fd-944e-b41f08b5903a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"
Sep 17 23:18:54.273094 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:54.273014    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"
Sep 17 23:18:54.547387 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:54.546478    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"]
Sep 17 23:18:54.932939 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:54.932902    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8" event={"ID":"5f47debb-3e28-49fd-944e-b41f08b5903a","Type":"ContainerStarted","Data":"cfd8bccfefb5236221294fa3539332013fceae62c9271682d050c4bd171b3119"}
Sep 17 23:18:56.948452 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:56.948369    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8" event={"ID":"5f47debb-3e28-49fd-944e-b41f08b5903a","Type":"ContainerStarted","Data":"a92452be304a8b8693870ba66543b39edc92f7ab2e1f1dd75b8ec03f2f4027d9"}
Sep 17 23:18:56.968545 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:56.968495    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8" podStartSLOduration=1.862636115 podStartE2EDuration="3.968480872s" totalImagesPullingTime="2.105844757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:18:56.965835487 +0000 UTC m=+756.970027307" watchObservedRunningTime="2026-09-17 23:18:56.968480872 +0000 UTC m=+756.972672711"
Sep 17 23:18:57.956209 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:57.956172    2581 generic.go:356] "Generic (PLEG): container finished" podID="5f47debb-3e28-49fd-944e-b41f08b5903a" containerID="a92452be304a8b8693870ba66543b39edc92f7ab2e1f1dd75b8ec03f2f4027d9" exitCode=0
Sep 17 23:18:57.956583 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:57.956259    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8" event={"ID":"5f47debb-3e28-49fd-944e-b41f08b5903a","Type":"ContainerDied","Data":"a92452be304a8b8693870ba66543b39edc92f7ab2e1f1dd75b8ec03f2f4027d9"}
Sep 17 23:18:59.108148 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:59.108115    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"
Sep 17 23:18:59.217389 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:59.217301    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f47debb-3e28-49fd-944e-b41f08b5903a-maas-api-service-ca\") pod \"5f47debb-3e28-49fd-944e-b41f08b5903a\" (UID: \"5f47debb-3e28-49fd-944e-b41f08b5903a\") "
Sep 17 23:18:59.217616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:59.217594    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f47debb-3e28-49fd-944e-b41f08b5903a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f47debb-3e28-49fd-944e-b41f08b5903a" (UID: "5f47debb-3e28-49fd-944e-b41f08b5903a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:18:59.217729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:59.217715    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f47debb-3e28-49fd-944e-b41f08b5903a-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:18:59.969859 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:59.969827    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"
Sep 17 23:18:59.970104 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:59.969852    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8" event={"ID":"5f47debb-3e28-49fd-944e-b41f08b5903a","Type":"ContainerDied","Data":"cfd8bccfefb5236221294fa3539332013fceae62c9271682d050c4bd171b3119"}
Sep 17 23:18:59.970104 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:18:59.969926    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfd8bccfefb5236221294fa3539332013fceae62c9271682d050c4bd171b3119"
Sep 17 23:19:08.024866 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.024829    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"]
Sep 17 23:19:08.025270 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.025260    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f47debb-3e28-49fd-944e-b41f08b5903a" containerName="revoke-keys"
Sep 17 23:19:08.025321 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.025272    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f47debb-3e28-49fd-944e-b41f08b5903a" containerName="revoke-keys"
Sep 17 23:19:08.025362 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.025339    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f47debb-3e28-49fd-944e-b41f08b5903a" containerName="revoke-keys"
Sep 17 23:19:08.028614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.028597    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.031315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.031290    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-6114ed\"/\"openshift-service-ca.crt\""
Sep 17 23:19:08.032435 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.032412    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-6114ed\"/\"autores-model-6114ed-kserve-self-signed-certs\""
Sep 17 23:19:08.032567 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.032446    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-6114ed\"/\"kube-root-ca.crt\""
Sep 17 23:19:08.032567 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.032465    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-6114ed\"/\"default-dockercfg-srkhh\""
Sep 17 23:19:08.039105 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.039081    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"]
Sep 17 23:19:08.210000 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.209959    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-home\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.210198 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.210099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78225\" (UniqueName: \"kubernetes.io/projected/119a7520-02b8-4248-abd4-fead879b3932-kube-api-access-78225\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.210198 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.210149    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-tmp-dir\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.210300 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.210211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-dshm\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.210300 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.210234    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-model-cache\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.210300 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.210261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/119a7520-02b8-4248-abd4-fead879b3932-tls-certs\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.311339 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.311243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78225\" (UniqueName: \"kubernetes.io/projected/119a7520-02b8-4248-abd4-fead879b3932-kube-api-access-78225\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.311339 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.311299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-tmp-dir\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.311564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.311362    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-dshm\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.311564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.311383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-model-cache\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.311564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.311419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/119a7520-02b8-4248-abd4-fead879b3932-tls-certs\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.311564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.311480    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-home\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.312090 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.312062    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-home\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.312688 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.312659    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-tmp-dir\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.317146 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.313295    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-model-cache\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.317146 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.316673    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/119a7520-02b8-4248-abd4-fead879b3932-tls-certs\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.321912 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.321882    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78225\" (UniqueName: \"kubernetes.io/projected/119a7520-02b8-4248-abd4-fead879b3932-kube-api-access-78225\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.326692 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.326629    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-dshm\") pod \"autores-model-6114ed-kserve-86df4bc77f-k4tx4\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") " pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.339332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.338974    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:08.615319 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:08.615286    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"]
Sep 17 23:19:08.616437 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:19:08.616403    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod119a7520_02b8_4248_abd4_fead879b3932.slice/crio-011d5350751729a3643eb7510d0c86b17a404dc7e0f74b07482b9fc2646dafe6 WatchSource:0}: Error finding container 011d5350751729a3643eb7510d0c86b17a404dc7e0f74b07482b9fc2646dafe6: Status 404 returned error can't find the container with id 011d5350751729a3643eb7510d0c86b17a404dc7e0f74b07482b9fc2646dafe6
Sep 17 23:19:09.022230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:09.022142    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4" event={"ID":"119a7520-02b8-4248-abd4-fead879b3932","Type":"ContainerStarted","Data":"ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283"}
Sep 17 23:19:09.022230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:09.022180    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4" event={"ID":"119a7520-02b8-4248-abd4-fead879b3932","Type":"ContainerStarted","Data":"011d5350751729a3643eb7510d0c86b17a404dc7e0f74b07482b9fc2646dafe6"}
Sep 17 23:19:09.041812 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:09.041757    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4" podStartSLOduration=1.041742713 podStartE2EDuration="1.041742713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:09.03946158 +0000 UTC m=+769.043653431" watchObservedRunningTime="2026-09-17 23:19:09.041742713 +0000 UTC m=+769.045934590"
Sep 17 23:19:10.886416 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:10.886383    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"]
Sep 17 23:19:10.886871 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:10.886751    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl" podUID="8dad70ff-d8c1-4734-b09b-c808429cc04f" containerName="payload-pre-processing" containerID="cri-o://c7c50b738d0dd49650436cc67cada5e2bdce3ebc1a4a437c2c84958683c2c3a1" gracePeriod=30
Sep 17 23:19:11.038472 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.038435    2581 generic.go:356] "Generic (PLEG): container finished" podID="8dad70ff-d8c1-4734-b09b-c808429cc04f" containerID="c7c50b738d0dd49650436cc67cada5e2bdce3ebc1a4a437c2c84958683c2c3a1" exitCode=0
Sep 17 23:19:11.038730 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.038697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl" event={"ID":"8dad70ff-d8c1-4734-b09b-c808429cc04f","Type":"ContainerDied","Data":"c7c50b738d0dd49650436cc67cada5e2bdce3ebc1a4a437c2c84958683c2c3a1"}
Sep 17 23:19:11.040324 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.040300    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"]
Sep 17 23:19:11.042860 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.042835    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b75118e5-bz4j8"]
Sep 17 23:19:11.196083 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.196060    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:19:11.237210 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.237179    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h5w6\" (UniqueName: \"kubernetes.io/projected/8dad70ff-d8c1-4734-b09b-c808429cc04f-kube-api-access-9h5w6\") pod \"8dad70ff-d8c1-4734-b09b-c808429cc04f\" (UID: \"8dad70ff-d8c1-4734-b09b-c808429cc04f\") "
Sep 17 23:19:11.237372 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.237240    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dad70ff-d8c1-4734-b09b-c808429cc04f-plugins-config-volume\") pod \"8dad70ff-d8c1-4734-b09b-c808429cc04f\" (UID: \"8dad70ff-d8c1-4734-b09b-c808429cc04f\") "
Sep 17 23:19:11.237678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.237645    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dad70ff-d8c1-4734-b09b-c808429cc04f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8dad70ff-d8c1-4734-b09b-c808429cc04f" (UID: "8dad70ff-d8c1-4734-b09b-c808429cc04f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:19:11.239727 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.239705    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dad70ff-d8c1-4734-b09b-c808429cc04f-kube-api-access-9h5w6" (OuterVolumeSpecName: "kube-api-access-9h5w6") pod "8dad70ff-d8c1-4734-b09b-c808429cc04f" (UID: "8dad70ff-d8c1-4734-b09b-c808429cc04f"). InnerVolumeSpecName "kube-api-access-9h5w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:19:11.338825 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.338785    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h5w6\" (UniqueName: \"kubernetes.io/projected/8dad70ff-d8c1-4734-b09b-c808429cc04f-kube-api-access-9h5w6\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:19:11.338825 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.338816    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dad70ff-d8c1-4734-b09b-c808429cc04f-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:19:11.994090 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:11.994057    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f47debb-3e28-49fd-944e-b41f08b5903a" path="/var/lib/kubelet/pods/5f47debb-3e28-49fd-944e-b41f08b5903a/volumes"
Sep 17 23:19:12.052151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:12.051147    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl" event={"ID":"8dad70ff-d8c1-4734-b09b-c808429cc04f","Type":"ContainerDied","Data":"c04b1b70ab5b964935ef719d967df6d5157b5997a2857b68bb4129d66c9fbc11"}
Sep 17 23:19:12.052151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:12.051215    2581 scope.go:117] "RemoveContainer" containerID="c7c50b738d0dd49650436cc67cada5e2bdce3ebc1a4a437c2c84958683c2c3a1"
Sep 17 23:19:12.052151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:12.051368    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"
Sep 17 23:19:12.078378 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:12.078321    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"]
Sep 17 23:19:12.081956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:12.081925    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b75118e5-5cc5d65c66-w7xxl"]
Sep 17 23:19:13.993958 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:13.993925    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dad70ff-d8c1-4734-b09b-c808429cc04f" path="/var/lib/kubelet/pods/8dad70ff-d8c1-4734-b09b-c808429cc04f/volumes"
Sep 17 23:19:17.759144 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.759020    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"]
Sep 17 23:19:17.760151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.759720    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dad70ff-d8c1-4734-b09b-c808429cc04f" containerName="payload-pre-processing"
Sep 17 23:19:17.760151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.759745    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dad70ff-d8c1-4734-b09b-c808429cc04f" containerName="payload-pre-processing"
Sep 17 23:19:17.760151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.759890    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dad70ff-d8c1-4734-b09b-c808429cc04f" containerName="payload-pre-processing"
Sep 17 23:19:17.768918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.766717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:17.769275 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.769244    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"]
Sep 17 23:19:17.770807 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.770200    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-92cb6c75-dockercfg-xvssz\""
Sep 17 23:19:17.770807 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.770495    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-92cb6c75\""
Sep 17 23:19:17.805549 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.805511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799b5d3d-d78d-4291-9fac-387dad1b1224-plugins-config-volume\") pod \"payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x\" (UID: \"799b5d3d-d78d-4291-9fac-387dad1b1224\") " pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:17.805719 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.805569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvfz7\" (UniqueName: \"kubernetes.io/projected/799b5d3d-d78d-4291-9fac-387dad1b1224-kube-api-access-qvfz7\") pod \"payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x\" (UID: \"799b5d3d-d78d-4291-9fac-387dad1b1224\") " pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:17.907246 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.907200    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvfz7\" (UniqueName: \"kubernetes.io/projected/799b5d3d-d78d-4291-9fac-387dad1b1224-kube-api-access-qvfz7\") pod \"payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x\" (UID: \"799b5d3d-d78d-4291-9fac-387dad1b1224\") " pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:17.907439 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.907368    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799b5d3d-d78d-4291-9fac-387dad1b1224-plugins-config-volume\") pod \"payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x\" (UID: \"799b5d3d-d78d-4291-9fac-387dad1b1224\") " pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:17.908077 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.908051    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799b5d3d-d78d-4291-9fac-387dad1b1224-plugins-config-volume\") pod \"payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x\" (UID: \"799b5d3d-d78d-4291-9fac-387dad1b1224\") " pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:17.923005 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:17.922968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvfz7\" (UniqueName: \"kubernetes.io/projected/799b5d3d-d78d-4291-9fac-387dad1b1224-kube-api-access-qvfz7\") pod \"payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x\" (UID: \"799b5d3d-d78d-4291-9fac-387dad1b1224\") " pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:18.084391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:18.084304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:18.340736 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:18.340630    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:18.340736 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:18.340701    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:18.357925 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:18.357897    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:18.373508 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:18.373481    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"]
Sep 17 23:19:18.374638 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:19:18.374614    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod799b5d3d_d78d_4291_9fac_387dad1b1224.slice/crio-0e60fe49d164680598706f555bb8c9cd1ce03d5a0e9e11551baa9286752bc0a3 WatchSource:0}: Error finding container 0e60fe49d164680598706f555bb8c9cd1ce03d5a0e9e11551baa9286752bc0a3: Status 404 returned error can't find the container with id 0e60fe49d164680598706f555bb8c9cd1ce03d5a0e9e11551baa9286752bc0a3
Sep 17 23:19:19.096083 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:19.096038    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x" event={"ID":"799b5d3d-d78d-4291-9fac-387dad1b1224","Type":"ContainerStarted","Data":"e19efae90daafeb8589da24951e6ee05bbadf919269f42576386c7f1e6aa6745"}
Sep 17 23:19:19.096476 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:19.096098    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x" event={"ID":"799b5d3d-d78d-4291-9fac-387dad1b1224","Type":"ContainerStarted","Data":"0e60fe49d164680598706f555bb8c9cd1ce03d5a0e9e11551baa9286752bc0a3"}
Sep 17 23:19:19.096476 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:19.096415    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:19.107796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:19.107764    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:19:19.116409 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:19.116341    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x" podStartSLOduration=2.116325828 podStartE2EDuration="2.116325828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:19.113054468 +0000 UTC m=+779.117246306" watchObservedRunningTime="2026-09-17 23:19:19.116325828 +0000 UTC m=+779.120517669"
Sep 17 23:19:30.104627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:30.104590    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:39.897076 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:39.896989    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"]
Sep 17 23:19:39.900830 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:39.900811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"
Sep 17 23:19:39.903728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:39.903704    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:19:39.903846 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:39.903705    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:19:39.908988 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:39.908949    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"]
Sep 17 23:19:40.041655 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:40.041618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47fb619f-0553-418d-92d8-660adbd3fd15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c\" (UID: \"47fb619f-0553-418d-92d8-660adbd3fd15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"
Sep 17 23:19:40.142619 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:40.142583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47fb619f-0553-418d-92d8-660adbd3fd15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c\" (UID: \"47fb619f-0553-418d-92d8-660adbd3fd15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"
Sep 17 23:19:40.143272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:40.143252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47fb619f-0553-418d-92d8-660adbd3fd15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c\" (UID: \"47fb619f-0553-418d-92d8-660adbd3fd15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"
Sep 17 23:19:40.214375 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:40.214292    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"
Sep 17 23:19:40.486832 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:40.486807    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"]
Sep 17 23:19:40.488260 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:19:40.488225    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47fb619f_0553_418d_92d8_660adbd3fd15.slice/crio-918c6e34ddfb0b7b688cacbea30f3d935bbc9752b2c6fda8ffa79761b0f7d387 WatchSource:0}: Error finding container 918c6e34ddfb0b7b688cacbea30f3d935bbc9752b2c6fda8ffa79761b0f7d387: Status 404 returned error can't find the container with id 918c6e34ddfb0b7b688cacbea30f3d935bbc9752b2c6fda8ffa79761b0f7d387
Sep 17 23:19:41.237521 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:41.237477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c" event={"ID":"47fb619f-0553-418d-92d8-660adbd3fd15","Type":"ContainerStarted","Data":"644a1c626bf199ef6e6d54c52aabd3eca2c1365869930768f280f00732c98c9f"}
Sep 17 23:19:41.237978 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:41.237551    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c" event={"ID":"47fb619f-0553-418d-92d8-660adbd3fd15","Type":"ContainerStarted","Data":"918c6e34ddfb0b7b688cacbea30f3d935bbc9752b2c6fda8ffa79761b0f7d387"}
Sep 17 23:19:41.255149 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:41.255079    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c" podStartSLOduration=2.255062414 podStartE2EDuration="2.255062414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:41.251703847 +0000 UTC m=+801.255895677" watchObservedRunningTime="2026-09-17 23:19:41.255062414 +0000 UTC m=+801.259254254"
Sep 17 23:19:42.245754 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:42.245719    2581 generic.go:356] "Generic (PLEG): container finished" podID="47fb619f-0553-418d-92d8-660adbd3fd15" containerID="644a1c626bf199ef6e6d54c52aabd3eca2c1365869930768f280f00732c98c9f" exitCode=0
Sep 17 23:19:42.246199 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:42.245807    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c" event={"ID":"47fb619f-0553-418d-92d8-660adbd3fd15","Type":"ContainerDied","Data":"644a1c626bf199ef6e6d54c52aabd3eca2c1365869930768f280f00732c98c9f"}
Sep 17 23:19:43.425985 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:43.425947    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"
Sep 17 23:19:43.474637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:43.474598    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47fb619f-0553-418d-92d8-660adbd3fd15-maas-api-service-ca\") pod \"47fb619f-0553-418d-92d8-660adbd3fd15\" (UID: \"47fb619f-0553-418d-92d8-660adbd3fd15\") "
Sep 17 23:19:43.475252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:43.475222    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47fb619f-0553-418d-92d8-660adbd3fd15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "47fb619f-0553-418d-92d8-660adbd3fd15" (UID: "47fb619f-0553-418d-92d8-660adbd3fd15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:19:43.576087 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:43.576012    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/47fb619f-0553-418d-92d8-660adbd3fd15-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:19:44.263225 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:44.263136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c" event={"ID":"47fb619f-0553-418d-92d8-660adbd3fd15","Type":"ContainerDied","Data":"918c6e34ddfb0b7b688cacbea30f3d935bbc9752b2c6fda8ffa79761b0f7d387"}
Sep 17 23:19:44.263225 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:44.263198    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="918c6e34ddfb0b7b688cacbea30f3d935bbc9752b2c6fda8ffa79761b0f7d387"
Sep 17 23:19:44.263225 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:44.263148    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"
Sep 17 23:19:47.984051 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:47.984019    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"]
Sep 17 23:19:49.042819 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.042784    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"]
Sep 17 23:19:49.043313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.043002    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x" podUID="799b5d3d-d78d-4291-9fac-387dad1b1224" containerName="payload-processing" containerID="cri-o://e19efae90daafeb8589da24951e6ee05bbadf919269f42576386c7f1e6aa6745" gracePeriod=30
Sep 17 23:19:49.200202 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.200164    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"]
Sep 17 23:19:49.202848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.202823    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-92cb6c75-kwk8c"]
Sep 17 23:19:49.295316 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.295246    2581 generic.go:356] "Generic (PLEG): container finished" podID="799b5d3d-d78d-4291-9fac-387dad1b1224" containerID="e19efae90daafeb8589da24951e6ee05bbadf919269f42576386c7f1e6aa6745" exitCode=0
Sep 17 23:19:49.295423 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.295324    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x" event={"ID":"799b5d3d-d78d-4291-9fac-387dad1b1224","Type":"ContainerDied","Data":"e19efae90daafeb8589da24951e6ee05bbadf919269f42576386c7f1e6aa6745"}
Sep 17 23:19:49.295423 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.295393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x" event={"ID":"799b5d3d-d78d-4291-9fac-387dad1b1224","Type":"ContainerDied","Data":"0e60fe49d164680598706f555bb8c9cd1ce03d5a0e9e11551baa9286752bc0a3"}
Sep 17 23:19:49.295423 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.295409    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e60fe49d164680598706f555bb8c9cd1ce03d5a0e9e11551baa9286752bc0a3"
Sep 17 23:19:49.305467 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.305449    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:49.338171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.338142    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799b5d3d-d78d-4291-9fac-387dad1b1224-plugins-config-volume\") pod \"799b5d3d-d78d-4291-9fac-387dad1b1224\" (UID: \"799b5d3d-d78d-4291-9fac-387dad1b1224\") "
Sep 17 23:19:49.338341 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.338261    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvfz7\" (UniqueName: \"kubernetes.io/projected/799b5d3d-d78d-4291-9fac-387dad1b1224-kube-api-access-qvfz7\") pod \"799b5d3d-d78d-4291-9fac-387dad1b1224\" (UID: \"799b5d3d-d78d-4291-9fac-387dad1b1224\") "
Sep 17 23:19:49.338609 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.338582    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/799b5d3d-d78d-4291-9fac-387dad1b1224-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "799b5d3d-d78d-4291-9fac-387dad1b1224" (UID: "799b5d3d-d78d-4291-9fac-387dad1b1224"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:19:49.340331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.340307    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/799b5d3d-d78d-4291-9fac-387dad1b1224-kube-api-access-qvfz7" (OuterVolumeSpecName: "kube-api-access-qvfz7") pod "799b5d3d-d78d-4291-9fac-387dad1b1224" (UID: "799b5d3d-d78d-4291-9fac-387dad1b1224"). InnerVolumeSpecName "kube-api-access-qvfz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:19:49.439786 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.439748    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvfz7\" (UniqueName: \"kubernetes.io/projected/799b5d3d-d78d-4291-9fac-387dad1b1224-kube-api-access-qvfz7\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:19:49.439786 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.439778    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799b5d3d-d78d-4291-9fac-387dad1b1224-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:19:49.993453 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:49.993418    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47fb619f-0553-418d-92d8-660adbd3fd15" path="/var/lib/kubelet/pods/47fb619f-0553-418d-92d8-660adbd3fd15/volumes"
Sep 17 23:19:50.301380 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:50.301295    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"
Sep 17 23:19:50.321476 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:50.321438    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"]
Sep 17 23:19:50.329686 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:50.329656    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-92cb6c75-8697448cb5-hgv6x"]
Sep 17 23:19:51.999922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:51.999887    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="799b5d3d-d78d-4291-9fac-387dad1b1224" path="/var/lib/kubelet/pods/799b5d3d-d78d-4291-9fac-387dad1b1224/volumes"
Sep 17 23:19:57.194579 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.194543    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"]
Sep 17 23:19:57.195017 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.194786    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw" podUID="08ee358c-688b-4218-8e59-b5b1d08c09ac" containerName="payload-processing" containerID="cri-o://d2f494a32a069ef06c3aa025a512d5ee8f6148caeaea6281d28ab5f5ce579843" gracePeriod=30
Sep 17 23:19:57.345306 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.345268    2581 generic.go:356] "Generic (PLEG): container finished" podID="08ee358c-688b-4218-8e59-b5b1d08c09ac" containerID="d2f494a32a069ef06c3aa025a512d5ee8f6148caeaea6281d28ab5f5ce579843" exitCode=0
Sep 17 23:19:57.345493 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.345326    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw" event={"ID":"08ee358c-688b-4218-8e59-b5b1d08c09ac","Type":"ContainerDied","Data":"d2f494a32a069ef06c3aa025a512d5ee8f6148caeaea6281d28ab5f5ce579843"}
Sep 17 23:19:57.456734 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.456656    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:19:57.522332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.522302    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ee358c-688b-4218-8e59-b5b1d08c09ac-plugins-config-volume\") pod \"08ee358c-688b-4218-8e59-b5b1d08c09ac\" (UID: \"08ee358c-688b-4218-8e59-b5b1d08c09ac\") "
Sep 17 23:19:57.522547 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.522357    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j625t\" (UniqueName: \"kubernetes.io/projected/08ee358c-688b-4218-8e59-b5b1d08c09ac-kube-api-access-j625t\") pod \"08ee358c-688b-4218-8e59-b5b1d08c09ac\" (UID: \"08ee358c-688b-4218-8e59-b5b1d08c09ac\") "
Sep 17 23:19:57.522710 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.522685    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08ee358c-688b-4218-8e59-b5b1d08c09ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "08ee358c-688b-4218-8e59-b5b1d08c09ac" (UID: "08ee358c-688b-4218-8e59-b5b1d08c09ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:19:57.524660 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.524634    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08ee358c-688b-4218-8e59-b5b1d08c09ac-kube-api-access-j625t" (OuterVolumeSpecName: "kube-api-access-j625t") pod "08ee358c-688b-4218-8e59-b5b1d08c09ac" (UID: "08ee358c-688b-4218-8e59-b5b1d08c09ac"). InnerVolumeSpecName "kube-api-access-j625t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:19:57.623476 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.623439    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j625t\" (UniqueName: \"kubernetes.io/projected/08ee358c-688b-4218-8e59-b5b1d08c09ac-kube-api-access-j625t\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:19:57.623476 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:57.623471    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08ee358c-688b-4218-8e59-b5b1d08c09ac-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:19:58.088091 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.088058    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"]
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089269    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="799b5d3d-d78d-4291-9fac-387dad1b1224" containerName="payload-processing"
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089300    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="799b5d3d-d78d-4291-9fac-387dad1b1224" containerName="payload-processing"
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089315    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47fb619f-0553-418d-92d8-660adbd3fd15" containerName="revoke-keys"
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089325    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="47fb619f-0553-418d-92d8-660adbd3fd15" containerName="revoke-keys"
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089398    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08ee358c-688b-4218-8e59-b5b1d08c09ac" containerName="payload-processing"
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089408    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="08ee358c-688b-4218-8e59-b5b1d08c09ac" containerName="payload-processing"
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089593    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="799b5d3d-d78d-4291-9fac-387dad1b1224" containerName="payload-processing"
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089605    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="47fb619f-0553-418d-92d8-660adbd3fd15" containerName="revoke-keys"
Sep 17 23:19:58.091150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.089628    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="08ee358c-688b-4218-8e59-b5b1d08c09ac" containerName="payload-processing"
Sep 17 23:19:58.093985 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.093959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:58.097558 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.097535    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-57a4f2d1-dockercfg-fclwl\""
Sep 17 23:19:58.097659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.097533    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-57a4f2d1\""
Sep 17 23:19:58.119430 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.119404    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"]
Sep 17 23:19:58.123342 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.123313    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"]
Sep 17 23:19:58.127580 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.127559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:58.137686 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.137662    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"]
Sep 17 23:19:58.231035 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.230984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzfp6\" (UniqueName: \"kubernetes.io/projected/20d11358-c57d-4f7a-b115-4563957f6e56-kube-api-access-zzfp6\") pod \"payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb\" (UID: \"20d11358-c57d-4f7a-b115-4563957f6e56\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:58.231509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.231041    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d11358-c57d-4f7a-b115-4563957f6e56-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb\" (UID: \"20d11358-c57d-4f7a-b115-4563957f6e56\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:58.231509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.231103    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s69q\" (UniqueName: \"kubernetes.io/projected/a797677e-ace9-4bde-956c-5d0daf8b2ca6-kube-api-access-6s69q\") pod \"payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw\" (UID: \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\") " pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:58.231509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.231181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a797677e-ace9-4bde-956c-5d0daf8b2ca6-plugins-config-volume\") pod \"payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw\" (UID: \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\") " pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:58.332190 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.332096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s69q\" (UniqueName: \"kubernetes.io/projected/a797677e-ace9-4bde-956c-5d0daf8b2ca6-kube-api-access-6s69q\") pod \"payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw\" (UID: \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\") " pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:58.332190 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.332184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a797677e-ace9-4bde-956c-5d0daf8b2ca6-plugins-config-volume\") pod \"payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw\" (UID: \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\") " pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:58.332405 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.332289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzfp6\" (UniqueName: \"kubernetes.io/projected/20d11358-c57d-4f7a-b115-4563957f6e56-kube-api-access-zzfp6\") pod \"payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb\" (UID: \"20d11358-c57d-4f7a-b115-4563957f6e56\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:58.332405 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.332312    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d11358-c57d-4f7a-b115-4563957f6e56-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb\" (UID: \"20d11358-c57d-4f7a-b115-4563957f6e56\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:58.332955 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.332934    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a797677e-ace9-4bde-956c-5d0daf8b2ca6-plugins-config-volume\") pod \"payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw\" (UID: \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\") " pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:58.333031 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.332938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d11358-c57d-4f7a-b115-4563957f6e56-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb\" (UID: \"20d11358-c57d-4f7a-b115-4563957f6e56\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:58.340623 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.340601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzfp6\" (UniqueName: \"kubernetes.io/projected/20d11358-c57d-4f7a-b115-4563957f6e56-kube-api-access-zzfp6\") pod \"payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb\" (UID: \"20d11358-c57d-4f7a-b115-4563957f6e56\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:58.340717 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.340604    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s69q\" (UniqueName: \"kubernetes.io/projected/a797677e-ace9-4bde-956c-5d0daf8b2ca6-kube-api-access-6s69q\") pod \"payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw\" (UID: \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\") " pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:58.353967 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.353946    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"
Sep 17 23:19:58.353967 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.353951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw" event={"ID":"08ee358c-688b-4218-8e59-b5b1d08c09ac","Type":"ContainerDied","Data":"906e1915451cccf91718651dcca772aafe9866f02c84e60af8fe11ecc92c0785"}
Sep 17 23:19:58.354222 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.354015    2581 scope.go:117] "RemoveContainer" containerID="d2f494a32a069ef06c3aa025a512d5ee8f6148caeaea6281d28ab5f5ce579843"
Sep 17 23:19:58.373346 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.373324    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"]
Sep 17 23:19:58.377831 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.377800    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-6114ed-55d5d84bc9-ftjcw"]
Sep 17 23:19:58.405911 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.405883    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:58.442188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.442154    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:58.689067 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.689040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"]
Sep 17 23:19:58.690273 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:19:58.690244    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d11358_c57d_4f7a_b115_4563957f6e56.slice/crio-6c0efe7537be3e028a6c59fbd2b512bd9f63085ca250ba7851487d4006d2c4cb WatchSource:0}: Error finding container 6c0efe7537be3e028a6c59fbd2b512bd9f63085ca250ba7851487d4006d2c4cb: Status 404 returned error can't find the container with id 6c0efe7537be3e028a6c59fbd2b512bd9f63085ca250ba7851487d4006d2c4cb
Sep 17 23:19:58.736308 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:58.736285    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"]
Sep 17 23:19:58.743237 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:19:58.743214    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda797677e_ace9_4bde_956c_5d0daf8b2ca6.slice/crio-f43c44451cada81696070c8aa719e334960921ecd6b1fea7894e2115b68381f8 WatchSource:0}: Error finding container f43c44451cada81696070c8aa719e334960921ecd6b1fea7894e2115b68381f8: Status 404 returned error can't find the container with id f43c44451cada81696070c8aa719e334960921ecd6b1fea7894e2115b68381f8
Sep 17 23:19:59.362489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.362449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw" event={"ID":"a797677e-ace9-4bde-956c-5d0daf8b2ca6","Type":"ContainerStarted","Data":"85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed"}
Sep 17 23:19:59.362951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.362578    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw" event={"ID":"a797677e-ace9-4bde-956c-5d0daf8b2ca6","Type":"ContainerStarted","Data":"f43c44451cada81696070c8aa719e334960921ecd6b1fea7894e2115b68381f8"}
Sep 17 23:19:59.362951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.362614    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:19:59.364432 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.364403    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb" event={"ID":"20d11358-c57d-4f7a-b115-4563957f6e56","Type":"ContainerStarted","Data":"2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8"}
Sep 17 23:19:59.364572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.364450    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb" event={"ID":"20d11358-c57d-4f7a-b115-4563957f6e56","Type":"ContainerStarted","Data":"6c0efe7537be3e028a6c59fbd2b512bd9f63085ca250ba7851487d4006d2c4cb"}
Sep 17 23:19:59.364572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.364481    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:19:59.382787 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.382741    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw" podStartSLOduration=1.382725547 podStartE2EDuration="1.382725547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:59.379651654 +0000 UTC m=+819.383843494" watchObservedRunningTime="2026-09-17 23:19:59.382725547 +0000 UTC m=+819.386917386"
Sep 17 23:19:59.399596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.399553    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb" podStartSLOduration=1.399539659 podStartE2EDuration="1.399539659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:19:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:19:59.397947028 +0000 UTC m=+819.402138866" watchObservedRunningTime="2026-09-17 23:19:59.399539659 +0000 UTC m=+819.403731497"
Sep 17 23:19:59.995443 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:19:59.995411    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08ee358c-688b-4218-8e59-b5b1d08c09ac" path="/var/lib/kubelet/pods/08ee358c-688b-4218-8e59-b5b1d08c09ac/volumes"
Sep 17 23:20:03.007429 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.007379    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4" podUID="119a7520-02b8-4248-abd4-fead879b3932" containerName="main" containerID="cri-o://ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283" gracePeriod=45
Sep 17 23:20:03.254501 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.254477    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:20:03.385728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.385701    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-home\") pod \"119a7520-02b8-4248-abd4-fead879b3932\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") "
Sep 17 23:20:03.385914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.385806    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78225\" (UniqueName: \"kubernetes.io/projected/119a7520-02b8-4248-abd4-fead879b3932-kube-api-access-78225\") pod \"119a7520-02b8-4248-abd4-fead879b3932\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") "
Sep 17 23:20:03.385914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.385858    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-dshm\") pod \"119a7520-02b8-4248-abd4-fead879b3932\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") "
Sep 17 23:20:03.385914 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.385902    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-model-cache\") pod \"119a7520-02b8-4248-abd4-fead879b3932\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") "
Sep 17 23:20:03.386066 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.385928    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/119a7520-02b8-4248-abd4-fead879b3932-tls-certs\") pod \"119a7520-02b8-4248-abd4-fead879b3932\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") "
Sep 17 23:20:03.386066 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.386000    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-tmp-dir\") pod \"119a7520-02b8-4248-abd4-fead879b3932\" (UID: \"119a7520-02b8-4248-abd4-fead879b3932\") "
Sep 17 23:20:03.386066 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.386010    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-home" (OuterVolumeSpecName: "home") pod "119a7520-02b8-4248-abd4-fead879b3932" (UID: "119a7520-02b8-4248-abd4-fead879b3932"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:03.386256 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.386222    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-model-cache" (OuterVolumeSpecName: "model-cache") pod "119a7520-02b8-4248-abd4-fead879b3932" (UID: "119a7520-02b8-4248-abd4-fead879b3932"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:03.386421 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.386389    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "119a7520-02b8-4248-abd4-fead879b3932" (UID: "119a7520-02b8-4248-abd4-fead879b3932"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:03.386540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.386490    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:03.386540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.386507    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:03.386540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.386520    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:03.388259 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.388235    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-dshm" (OuterVolumeSpecName: "dshm") pod "119a7520-02b8-4248-abd4-fead879b3932" (UID: "119a7520-02b8-4248-abd4-fead879b3932"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:03.388360 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.388298    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119a7520-02b8-4248-abd4-fead879b3932-kube-api-access-78225" (OuterVolumeSpecName: "kube-api-access-78225") pod "119a7520-02b8-4248-abd4-fead879b3932" (UID: "119a7520-02b8-4248-abd4-fead879b3932"). InnerVolumeSpecName "kube-api-access-78225". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:20:03.388431 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.388384    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/119a7520-02b8-4248-abd4-fead879b3932-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "119a7520-02b8-4248-abd4-fead879b3932" (UID: "119a7520-02b8-4248-abd4-fead879b3932"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:20:03.391631 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.391608    2581 generic.go:356] "Generic (PLEG): container finished" podID="119a7520-02b8-4248-abd4-fead879b3932" containerID="ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283" exitCode=0
Sep 17 23:20:03.391737 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.391646    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"
Sep 17 23:20:03.391737 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.391686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4" event={"ID":"119a7520-02b8-4248-abd4-fead879b3932","Type":"ContainerDied","Data":"ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283"}
Sep 17 23:20:03.391737 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.391730    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4" event={"ID":"119a7520-02b8-4248-abd4-fead879b3932","Type":"ContainerDied","Data":"011d5350751729a3643eb7510d0c86b17a404dc7e0f74b07482b9fc2646dafe6"}
Sep 17 23:20:03.391912 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.391752    2581 scope.go:117] "RemoveContainer" containerID="ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283"
Sep 17 23:20:03.408999 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.408979    2581 scope.go:117] "RemoveContainer" containerID="ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283"
Sep 17 23:20:03.409363 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:20:03.409342    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283\": container with ID starting with ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283 not found: ID does not exist" containerID="ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283"
Sep 17 23:20:03.409441 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.409372    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283"} err="failed to get container status \"ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283\": rpc error: code = NotFound desc = could not find container \"ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283\": container with ID starting with ce53966de569e0ed5d0a58f931af59e24c74543e51b94f9bb2abf6ac87d08283 not found: ID does not exist"
Sep 17 23:20:03.420416 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.420393    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"]
Sep 17 23:20:03.425827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.425799    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-6114ed/autores-model-6114ed-kserve-86df4bc77f-k4tx4"]
Sep 17 23:20:03.487870 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.487844    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78225\" (UniqueName: \"kubernetes.io/projected/119a7520-02b8-4248-abd4-fead879b3932-kube-api-access-78225\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:03.487870 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.487866    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/119a7520-02b8-4248-abd4-fead879b3932-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:03.488038 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.487877    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/119a7520-02b8-4248-abd4-fead879b3932-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:03.994160 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:03.994109    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="119a7520-02b8-4248-abd4-fead879b3932" path="/var/lib/kubelet/pods/119a7520-02b8-4248-abd4-fead879b3932/volumes"
Sep 17 23:20:09.082787 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:09.082745    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"]
Sep 17 23:20:09.083259 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:09.083014    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy" containerID="cri-o://20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb" gracePeriod=30
Sep 17 23:20:10.375042 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:10.375005    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:20:10.375496 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:10.375185    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:20:11.825216 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.825180    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"]
Sep 17 23:20:11.825654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.825596    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="119a7520-02b8-4248-abd4-fead879b3932" containerName="main"
Sep 17 23:20:11.825654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.825607    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="119a7520-02b8-4248-abd4-fead879b3932" containerName="main"
Sep 17 23:20:11.825727 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.825673    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="119a7520-02b8-4248-abd4-fead879b3932" containerName="main"
Sep 17 23:20:11.828843 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.828827    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:11.831530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.831504    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-7ab633\"/\"kube-root-ca.crt\""
Sep 17 23:20:11.831671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.831543    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-7ab633\"/\"default-dockercfg-b9f5k\""
Sep 17 23:20:11.837141 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.833304    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-7ab633\"/\"openshift-service-ca.crt\""
Sep 17 23:20:11.837141 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.833330    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-7ab633\"/\"premium-w4-7ab633-kserve-self-signed-certs\""
Sep 17 23:20:11.841302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.841277    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"]
Sep 17 23:20:11.971030 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.970998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntx5k\" (UniqueName: \"kubernetes.io/projected/87802f0c-c645-4ef9-8d8a-abb56d55d89e-kube-api-access-ntx5k\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:11.971228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.971040    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-home\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:11.971228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.971088    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tls-certs\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:11.971228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.971141    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-dshm\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:11.971228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.971180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tmp-dir\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:11.971228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:11.971215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-model-cache\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.072103 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.072064    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntx5k\" (UniqueName: \"kubernetes.io/projected/87802f0c-c645-4ef9-8d8a-abb56d55d89e-kube-api-access-ntx5k\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.072315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.072151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-home\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.072315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.072220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tls-certs\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.072315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.072249    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-dshm\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.072315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.072300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tmp-dir\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.072551 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.072347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-model-cache\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.072878 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.072854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-home\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.073043 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.073002    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tmp-dir\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.073160 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.073055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-model-cache\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.074748 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.074726    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-dshm\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.075014 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.074993    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tls-certs\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.081794 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.081716    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntx5k\" (UniqueName: \"kubernetes.io/projected/87802f0c-c645-4ef9-8d8a-abb56d55d89e-kube-api-access-ntx5k\") pod \"premium-w4-7ab633-kserve-7bf496c678-v9dfn\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") " pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.145470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.145436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:12.448487 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:12.448454    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"]
Sep 17 23:20:12.449590 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:20:12.449563    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87802f0c_c645_4ef9_8d8a_abb56d55d89e.slice/crio-c12044a85a1bd7ea647b48d60dfb636bd3eae94c89efc167fae5c6b57a6c6f17 WatchSource:0}: Error finding container c12044a85a1bd7ea647b48d60dfb636bd3eae94c89efc167fae5c6b57a6c6f17: Status 404 returned error can't find the container with id c12044a85a1bd7ea647b48d60dfb636bd3eae94c89efc167fae5c6b57a6c6f17
Sep 17 23:20:13.129406 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.129368    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"]
Sep 17 23:20:13.133279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.133254    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"
Sep 17 23:20:13.135939 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.135922    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:20:13.136039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.135924    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:20:13.142072 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.142045    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"]
Sep 17 23:20:13.196936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.196903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ae939-7368-4021-903a-fce192e63d5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf\" (UID: \"533ae939-7368-4021-903a-fce192e63d5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"
Sep 17 23:20:13.297646 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.297607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ae939-7368-4021-903a-fce192e63d5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf\" (UID: \"533ae939-7368-4021-903a-fce192e63d5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"
Sep 17 23:20:13.298267 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.298249    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ae939-7368-4021-903a-fce192e63d5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf\" (UID: \"533ae939-7368-4021-903a-fce192e63d5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"
Sep 17 23:20:13.445668 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.445581    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"
Sep 17 23:20:13.461738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.461705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn" event={"ID":"87802f0c-c645-4ef9-8d8a-abb56d55d89e","Type":"ContainerStarted","Data":"3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43"}
Sep 17 23:20:13.461868 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.461756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn" event={"ID":"87802f0c-c645-4ef9-8d8a-abb56d55d89e","Type":"ContainerStarted","Data":"c12044a85a1bd7ea647b48d60dfb636bd3eae94c89efc167fae5c6b57a6c6f17"}
Sep 17 23:20:13.486181 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.486096    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn" podStartSLOduration=2.486073009 podStartE2EDuration="2.486073009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:20:13.481942927 +0000 UTC m=+833.486134769" watchObservedRunningTime="2026-09-17 23:20:13.486073009 +0000 UTC m=+833.490264849"
Sep 17 23:20:13.727272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:13.727248    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"]
Sep 17 23:20:13.728410 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:20:13.728388    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533ae939_7368_4021_903a_fce192e63d5e.slice/crio-6770c45fb4ad0dcec677b13675f6f3e3cd7286a6f72ce4c7ad18ad3396117d6d WatchSource:0}: Error finding container 6770c45fb4ad0dcec677b13675f6f3e3cd7286a6f72ce4c7ad18ad3396117d6d: Status 404 returned error can't find the container with id 6770c45fb4ad0dcec677b13675f6f3e3cd7286a6f72ce4c7ad18ad3396117d6d
Sep 17 23:20:14.342472 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.342450    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:20:14.410882 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.410852    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p2b8\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-kube-api-access-7p2b8\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.410892    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-socket\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.410912    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-credential-socket\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.410942    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-data\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.410973    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-certs\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411029    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-podinfo\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411057    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-token\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411102    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-envoy\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411214    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd8809bb-864e-4aa6-b44b-e32f268bc647-istiod-ca-cert\") pod \"dd8809bb-864e-4aa6-b44b-e32f268bc647\" (UID: \"dd8809bb-864e-4aa6-b44b-e32f268bc647\") "
Sep 17 23:20:14.411335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411281    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:14.411335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411331    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:14.411603 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411336    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:14.411664 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411609    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-credential-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.411664 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411628    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-data\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.411664 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411640    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.411664 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411634    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:14.411881 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.411662    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd8809bb-864e-4aa6-b44b-e32f268bc647-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:20:14.413308 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.413282    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:20:14.413394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.413355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:20:14.413553 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.413538    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-kube-api-access-7p2b8" (OuterVolumeSpecName: "kube-api-access-7p2b8") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "kube-api-access-7p2b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:20:14.413661 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.413640    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd8809bb-864e-4aa6-b44b-e32f268bc647" (UID: "dd8809bb-864e-4aa6-b44b-e32f268bc647"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:20:14.471636 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.471606    2581 generic.go:356] "Generic (PLEG): container finished" podID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerID="20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb" exitCode=0
Sep 17 23:20:14.471790 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.471663    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"
Sep 17 23:20:14.471790 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.471686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" event={"ID":"dd8809bb-864e-4aa6-b44b-e32f268bc647","Type":"ContainerDied","Data":"20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb"}
Sep 17 23:20:14.471790 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.471735    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4" event={"ID":"dd8809bb-864e-4aa6-b44b-e32f268bc647","Type":"ContainerDied","Data":"8b262bbcb2f988de4c9cbfd059c49e5acb6384cb09ef632b1d0188439b476a54"}
Sep 17 23:20:14.471790 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.471758    2581 scope.go:117] "RemoveContainer" containerID="20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb"
Sep 17 23:20:14.473482 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.473464    2581 generic.go:356] "Generic (PLEG): container finished" podID="533ae939-7368-4021-903a-fce192e63d5e" containerID="b3a5d2c685284c94686c1b4dcc82bef4c60bcc275609c581818f3598e83270c5" exitCode=0
Sep 17 23:20:14.473575 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.473544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf" event={"ID":"533ae939-7368-4021-903a-fce192e63d5e","Type":"ContainerDied","Data":"b3a5d2c685284c94686c1b4dcc82bef4c60bcc275609c581818f3598e83270c5"}
Sep 17 23:20:14.473633 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.473580    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf" event={"ID":"533ae939-7368-4021-903a-fce192e63d5e","Type":"ContainerStarted","Data":"6770c45fb4ad0dcec677b13675f6f3e3cd7286a6f72ce4c7ad18ad3396117d6d"}
Sep 17 23:20:14.512510 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.512479    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-podinfo\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.512510 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.512509    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-token\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.512815 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.512521    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-istio-envoy\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.512815 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.512528    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd8809bb-864e-4aa6-b44b-e32f268bc647-istiod-ca-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.512815 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.512538    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p2b8\" (UniqueName: \"kubernetes.io/projected/dd8809bb-864e-4aa6-b44b-e32f268bc647-kube-api-access-7p2b8\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.512815 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.512556    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd8809bb-864e-4aa6-b44b-e32f268bc647-workload-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:14.521008 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.520991    2581 scope.go:117] "RemoveContainer" containerID="279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08"
Sep 17 23:20:14.530253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.530210    2581 scope.go:117] "RemoveContainer" containerID="20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb"
Sep 17 23:20:14.530559 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:20:14.530539    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb\": container with ID starting with 20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb not found: ID does not exist" containerID="20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb"
Sep 17 23:20:14.530639 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.530587    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb"} err="failed to get container status \"20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb\": rpc error: code = NotFound desc = could not find container \"20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb\": container with ID starting with 20cef332f48e293b3493ecf163acf5d9b3277db1d71119015b99ad26bad726fb not found: ID does not exist"
Sep 17 23:20:14.530710 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.530639    2581 scope.go:117] "RemoveContainer" containerID="279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08"
Sep 17 23:20:14.530964 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:20:14.530945    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08\": container with ID starting with 279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08 not found: ID does not exist" containerID="279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08"
Sep 17 23:20:14.531008 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.530972    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08"} err="failed to get container status \"279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08\": rpc error: code = NotFound desc = could not find container \"279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08\": container with ID starting with 279ee15e0c381694594c934f4da51463543d57c0d8ed274b10e415746108ac08 not found: ID does not exist"
Sep 17 23:20:14.533449 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.533430    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"]
Sep 17 23:20:14.538265 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:14.538245    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-6114ed-openshift-default-788fc9544-bsrf4"]
Sep 17 23:20:15.630946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:15.630911    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"
Sep 17 23:20:15.722407 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:15.722374    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ae939-7368-4021-903a-fce192e63d5e-maas-api-service-ca\") pod \"533ae939-7368-4021-903a-fce192e63d5e\" (UID: \"533ae939-7368-4021-903a-fce192e63d5e\") "
Sep 17 23:20:15.722800 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:15.722779    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/533ae939-7368-4021-903a-fce192e63d5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "533ae939-7368-4021-903a-fce192e63d5e" (UID: "533ae939-7368-4021-903a-fce192e63d5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:20:15.824012 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:15.823926    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/533ae939-7368-4021-903a-fce192e63d5e-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:15.994637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:15.994603    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" path="/var/lib/kubelet/pods/dd8809bb-864e-4aa6-b44b-e32f268bc647/volumes"
Sep 17 23:20:16.489222 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:16.489194    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"
Sep 17 23:20:16.489387 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:16.489237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf" event={"ID":"533ae939-7368-4021-903a-fce192e63d5e","Type":"ContainerDied","Data":"6770c45fb4ad0dcec677b13675f6f3e3cd7286a6f72ce4c7ad18ad3396117d6d"}
Sep 17 23:20:16.489387 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:16.489291    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6770c45fb4ad0dcec677b13675f6f3e3cd7286a6f72ce4c7ad18ad3396117d6d"
Sep 17 23:20:18.095115 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095087    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"]
Sep 17 23:20:18.095711 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095690    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533ae939-7368-4021-903a-fce192e63d5e" containerName="revoke-keys"
Sep 17 23:20:18.095818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095713    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="533ae939-7368-4021-903a-fce192e63d5e" containerName="revoke-keys"
Sep 17 23:20:18.095818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095728    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy"
Sep 17 23:20:18.095818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095736    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy"
Sep 17 23:20:18.095818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095749    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy"
Sep 17 23:20:18.095818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095756    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy"
Sep 17 23:20:18.096054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095873    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="533ae939-7368-4021-903a-fce192e63d5e" containerName="revoke-keys"
Sep 17 23:20:18.096054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095891    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy"
Sep 17 23:20:18.096054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.095901    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd8809bb-864e-4aa6-b44b-e32f268bc647" containerName="istio-proxy"
Sep 17 23:20:18.099286 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.099265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:18.103022 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.103001    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-f324fe\""
Sep 17 23:20:18.110929 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.110859    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"]
Sep 17 23:20:18.138777 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.138749    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"]
Sep 17 23:20:18.142468 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.142445    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:18.145246 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.145225    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-f324fe-dockercfg-sxwsp\""
Sep 17 23:20:18.147740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.147717    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5dgk\" (UniqueName: \"kubernetes.io/projected/6028c136-c76a-4a0d-9d08-551015e7c5c9-kube-api-access-z5dgk\") pod \"payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc\" (UID: \"6028c136-c76a-4a0d-9d08-551015e7c5c9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:18.147834 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.147809    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6028c136-c76a-4a0d-9d08-551015e7c5c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc\" (UID: \"6028c136-c76a-4a0d-9d08-551015e7c5c9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:18.153875 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.153853    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"]
Sep 17 23:20:18.249988 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.248458    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjg8c\" (UniqueName: \"kubernetes.io/projected/def99496-e8c2-401c-a3ec-02f5b3b3aef9-kube-api-access-mjg8c\") pod \"payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl\" (UID: \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\") " pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:18.249988 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.248596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5dgk\" (UniqueName: \"kubernetes.io/projected/6028c136-c76a-4a0d-9d08-551015e7c5c9-kube-api-access-z5dgk\") pod \"payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc\" (UID: \"6028c136-c76a-4a0d-9d08-551015e7c5c9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:18.249988 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.248658    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def99496-e8c2-401c-a3ec-02f5b3b3aef9-plugins-config-volume\") pod \"payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl\" (UID: \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\") " pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:18.249988 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.248700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6028c136-c76a-4a0d-9d08-551015e7c5c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc\" (UID: \"6028c136-c76a-4a0d-9d08-551015e7c5c9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:18.249988 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.249622    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6028c136-c76a-4a0d-9d08-551015e7c5c9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc\" (UID: \"6028c136-c76a-4a0d-9d08-551015e7c5c9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:18.261047 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.260986    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5dgk\" (UniqueName: \"kubernetes.io/projected/6028c136-c76a-4a0d-9d08-551015e7c5c9-kube-api-access-z5dgk\") pod \"payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc\" (UID: \"6028c136-c76a-4a0d-9d08-551015e7c5c9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:18.349924 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.349838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def99496-e8c2-401c-a3ec-02f5b3b3aef9-plugins-config-volume\") pod \"payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl\" (UID: \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\") " pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:18.349924 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.349907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjg8c\" (UniqueName: \"kubernetes.io/projected/def99496-e8c2-401c-a3ec-02f5b3b3aef9-kube-api-access-mjg8c\") pod \"payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl\" (UID: \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\") " pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:18.350627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.350606    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def99496-e8c2-401c-a3ec-02f5b3b3aef9-plugins-config-volume\") pod \"payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl\" (UID: \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\") " pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:18.358034 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.358003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjg8c\" (UniqueName: \"kubernetes.io/projected/def99496-e8c2-401c-a3ec-02f5b3b3aef9-kube-api-access-mjg8c\") pod \"payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl\" (UID: \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\") " pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:18.412018 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.411981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:18.455601 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.455561    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:18.695448 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:20:18.695417    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6028c136_c76a_4a0d_9d08_551015e7c5c9.slice/crio-5434afd6ad2c6bc2c46bbfd4bbe61e34b26ba96a17fbc57d991adced76d6287d WatchSource:0}: Error finding container 5434afd6ad2c6bc2c46bbfd4bbe61e34b26ba96a17fbc57d991adced76d6287d: Status 404 returned error can't find the container with id 5434afd6ad2c6bc2c46bbfd4bbe61e34b26ba96a17fbc57d991adced76d6287d
Sep 17 23:20:18.696494 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.696436    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"]
Sep 17 23:20:18.760943 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:18.760914    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"]
Sep 17 23:20:18.762397 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:20:18.762372    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef99496_e8c2_401c_a3ec_02f5b3b3aef9.slice/crio-3dfd1993a8627fbd1c118415b108ca5f2299746e9ab54646c476c444e3bc1436 WatchSource:0}: Error finding container 3dfd1993a8627fbd1c118415b108ca5f2299746e9ab54646c476c444e3bc1436: Status 404 returned error can't find the container with id 3dfd1993a8627fbd1c118415b108ca5f2299746e9ab54646c476c444e3bc1436
Sep 17 23:20:19.513513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:19.513467    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc" event={"ID":"6028c136-c76a-4a0d-9d08-551015e7c5c9","Type":"ContainerStarted","Data":"9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9"}
Sep 17 23:20:19.514203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:19.514177    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc" event={"ID":"6028c136-c76a-4a0d-9d08-551015e7c5c9","Type":"ContainerStarted","Data":"5434afd6ad2c6bc2c46bbfd4bbe61e34b26ba96a17fbc57d991adced76d6287d"}
Sep 17 23:20:19.514393 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:19.514378    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:20:19.515588 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:19.515565    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl" event={"ID":"def99496-e8c2-401c-a3ec-02f5b3b3aef9","Type":"ContainerStarted","Data":"d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5"}
Sep 17 23:20:19.515708 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:19.515611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl" event={"ID":"def99496-e8c2-401c-a3ec-02f5b3b3aef9","Type":"ContainerStarted","Data":"3dfd1993a8627fbd1c118415b108ca5f2299746e9ab54646c476c444e3bc1436"}
Sep 17 23:20:19.515708 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:19.515671    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:19.532320 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:19.532256    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc" podStartSLOduration=1.532238556 podStartE2EDuration="1.532238556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:20:19.52985596 +0000 UTC m=+839.534047796" watchObservedRunningTime="2026-09-17 23:20:19.532238556 +0000 UTC m=+839.536430400"
Sep 17 23:20:19.551555 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:19.551505    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl" podStartSLOduration=1.55149034 podStartE2EDuration="1.55149034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:20:19.549096362 +0000 UTC m=+839.553288205" watchObservedRunningTime="2026-09-17 23:20:19.55149034 +0000 UTC m=+839.555682180"
Sep 17 23:20:22.146182 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:22.146149    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:22.146583 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:22.146195    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:22.158800 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:22.158776    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:22.548501 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:22.548471    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:20:23.809850 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:23.809803    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"]
Sep 17 23:20:23.810328 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:23.810091    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw" podUID="a797677e-ace9-4bde-956c-5d0daf8b2ca6" containerName="payload-processing" containerID="cri-o://85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed" gracePeriod=30
Sep 17 23:20:23.813878 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:23.813851    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"]
Sep 17 23:20:23.814218 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:23.814068    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb" podUID="20d11358-c57d-4f7a-b115-4563957f6e56" containerName="payload-pre-processing" containerID="cri-o://2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8" gracePeriod=30
Sep 17 23:20:23.967649 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:23.967566    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"]
Sep 17 23:20:23.971689 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:23.971658    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-57a4f2d1-cz4rf"]
Sep 17 23:20:23.994660 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:23.994631    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="533ae939-7368-4021-903a-fce192e63d5e" path="/var/lib/kubelet/pods/533ae939-7368-4021-903a-fce192e63d5e/volumes"
Sep 17 23:20:24.136306 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.136273    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:20:24.138368 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.138349    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:20:24.213151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.212482    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzfp6\" (UniqueName: \"kubernetes.io/projected/20d11358-c57d-4f7a-b115-4563957f6e56-kube-api-access-zzfp6\") pod \"20d11358-c57d-4f7a-b115-4563957f6e56\" (UID: \"20d11358-c57d-4f7a-b115-4563957f6e56\") "
Sep 17 23:20:24.213151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.212585    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6s69q\" (UniqueName: \"kubernetes.io/projected/a797677e-ace9-4bde-956c-5d0daf8b2ca6-kube-api-access-6s69q\") pod \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\" (UID: \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\") "
Sep 17 23:20:24.213151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.212653    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a797677e-ace9-4bde-956c-5d0daf8b2ca6-plugins-config-volume\") pod \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\" (UID: \"a797677e-ace9-4bde-956c-5d0daf8b2ca6\") "
Sep 17 23:20:24.213151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.212700    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d11358-c57d-4f7a-b115-4563957f6e56-plugins-config-volume\") pod \"20d11358-c57d-4f7a-b115-4563957f6e56\" (UID: \"20d11358-c57d-4f7a-b115-4563957f6e56\") "
Sep 17 23:20:24.213749 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.213533    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d11358-c57d-4f7a-b115-4563957f6e56-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20d11358-c57d-4f7a-b115-4563957f6e56" (UID: "20d11358-c57d-4f7a-b115-4563957f6e56"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:20:24.214315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.214276    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a797677e-ace9-4bde-956c-5d0daf8b2ca6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a797677e-ace9-4bde-956c-5d0daf8b2ca6" (UID: "a797677e-ace9-4bde-956c-5d0daf8b2ca6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:20:24.222212 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.216337    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d11358-c57d-4f7a-b115-4563957f6e56-kube-api-access-zzfp6" (OuterVolumeSpecName: "kube-api-access-zzfp6") pod "20d11358-c57d-4f7a-b115-4563957f6e56" (UID: "20d11358-c57d-4f7a-b115-4563957f6e56"). InnerVolumeSpecName "kube-api-access-zzfp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:20:24.226625 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.226576    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a797677e-ace9-4bde-956c-5d0daf8b2ca6-kube-api-access-6s69q" (OuterVolumeSpecName: "kube-api-access-6s69q") pod "a797677e-ace9-4bde-956c-5d0daf8b2ca6" (UID: "a797677e-ace9-4bde-956c-5d0daf8b2ca6"). InnerVolumeSpecName "kube-api-access-6s69q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:20:24.314348 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.314316    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzfp6\" (UniqueName: \"kubernetes.io/projected/20d11358-c57d-4f7a-b115-4563957f6e56-kube-api-access-zzfp6\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:24.314348 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.314342    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6s69q\" (UniqueName: \"kubernetes.io/projected/a797677e-ace9-4bde-956c-5d0daf8b2ca6-kube-api-access-6s69q\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:24.314348 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.314352    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a797677e-ace9-4bde-956c-5d0daf8b2ca6-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:24.314577 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.314363    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d11358-c57d-4f7a-b115-4563957f6e56-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:20:24.551587 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.551550    2581 generic.go:356] "Generic (PLEG): container finished" podID="a797677e-ace9-4bde-956c-5d0daf8b2ca6" containerID="85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed" exitCode=0
Sep 17 23:20:24.551785 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.551598    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"
Sep 17 23:20:24.551785 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.551634    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw" event={"ID":"a797677e-ace9-4bde-956c-5d0daf8b2ca6","Type":"ContainerDied","Data":"85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed"}
Sep 17 23:20:24.551785 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.551684    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw" event={"ID":"a797677e-ace9-4bde-956c-5d0daf8b2ca6","Type":"ContainerDied","Data":"f43c44451cada81696070c8aa719e334960921ecd6b1fea7894e2115b68381f8"}
Sep 17 23:20:24.551785 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.551706    2581 scope.go:117] "RemoveContainer" containerID="85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed"
Sep 17 23:20:24.553184 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.553152    2581 generic.go:356] "Generic (PLEG): container finished" podID="20d11358-c57d-4f7a-b115-4563957f6e56" containerID="2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8" exitCode=0
Sep 17 23:20:24.553321 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.553204    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"
Sep 17 23:20:24.553321 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.553226    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb" event={"ID":"20d11358-c57d-4f7a-b115-4563957f6e56","Type":"ContainerDied","Data":"2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8"}
Sep 17 23:20:24.553321 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.553283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb" event={"ID":"20d11358-c57d-4f7a-b115-4563957f6e56","Type":"ContainerDied","Data":"6c0efe7537be3e028a6c59fbd2b512bd9f63085ca250ba7851487d4006d2c4cb"}
Sep 17 23:20:24.562824 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.562805    2581 scope.go:117] "RemoveContainer" containerID="85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed"
Sep 17 23:20:24.563149 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:20:24.563106    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed\": container with ID starting with 85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed not found: ID does not exist" containerID="85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed"
Sep 17 23:20:24.563224 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.563145    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed"} err="failed to get container status \"85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed\": rpc error: code = NotFound desc = could not find container \"85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed\": container with ID starting with 85f712f15d11f19aff244cd5f40fd8ac43d74abbbca5ab82c8de59ae861c37ed not found: ID does not exist"
Sep 17 23:20:24.563224 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.563165    2581 scope.go:117] "RemoveContainer" containerID="2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8"
Sep 17 23:20:24.572383 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.572368    2581 scope.go:117] "RemoveContainer" containerID="2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8"
Sep 17 23:20:24.572657 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:20:24.572637    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8\": container with ID starting with 2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8 not found: ID does not exist" containerID="2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8"
Sep 17 23:20:24.572724 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.572664    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8"} err="failed to get container status \"2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8\": rpc error: code = NotFound desc = could not find container \"2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8\": container with ID starting with 2d713e28c8bf7fd506a117508b387faaf2d7ef87c87b26b4aa395af5123583f8 not found: ID does not exist"
Sep 17 23:20:24.574532 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.574512    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"]
Sep 17 23:20:24.578926 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.578903    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-57a4f2d1-7564f488b9-tckdw"]
Sep 17 23:20:24.591598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.591567    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"]
Sep 17 23:20:24.593996 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:24.593977    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-57a4f2d1-5f5499789d-zzfqb"]
Sep 17 23:20:25.995074 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:25.995030    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20d11358-c57d-4f7a-b115-4563957f6e56" path="/var/lib/kubelet/pods/20d11358-c57d-4f7a-b115-4563957f6e56/volumes"
Sep 17 23:20:25.995644 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:25.995622    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a797677e-ace9-4bde-956c-5d0daf8b2ca6" path="/var/lib/kubelet/pods/a797677e-ace9-4bde-956c-5d0daf8b2ca6/volumes"
Sep 17 23:20:30.524070 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:30.524040    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:20:31.530501 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:20:31.530470    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:21:15.044262 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.044228    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"]
Sep 17 23:21:15.044789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.044634    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a797677e-ace9-4bde-956c-5d0daf8b2ca6" containerName="payload-processing"
Sep 17 23:21:15.044789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.044645    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a797677e-ace9-4bde-956c-5d0daf8b2ca6" containerName="payload-processing"
Sep 17 23:21:15.044789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.044664    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20d11358-c57d-4f7a-b115-4563957f6e56" containerName="payload-pre-processing"
Sep 17 23:21:15.044789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.044671    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="20d11358-c57d-4f7a-b115-4563957f6e56" containerName="payload-pre-processing"
Sep 17 23:21:15.044789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.044741    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a797677e-ace9-4bde-956c-5d0daf8b2ca6" containerName="payload-processing"
Sep 17 23:21:15.044789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.044753    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="20d11358-c57d-4f7a-b115-4563957f6e56" containerName="payload-pre-processing"
Sep 17 23:21:15.047731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.047713    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.050309 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.050287    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-f324fe\"/\"openshift-service-ca.crt\""
Sep 17 23:21:15.051403 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.051386    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-f324fe\"/\"default-dockercfg-cgs8m\""
Sep 17 23:21:15.051504 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.051437    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-f324fe\"/\"kube-root-ca.crt\""
Sep 17 23:21:15.051504 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.051392    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-f324fe\"/\"explicit-model-f324fe-kserve-self-signed-certs\""
Sep 17 23:21:15.058567 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.058544    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"]
Sep 17 23:21:15.102764 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.102734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-dshm\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.102917 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.102793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-home\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.102917 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.102840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-tmp-dir\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.102917 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.102904    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-model-cache\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.103056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.102942    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tqj5\" (UniqueName: \"kubernetes.io/projected/b05e2863-100a-447d-80a5-57de29631180-kube-api-access-7tqj5\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.103056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.103032    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05e2863-100a-447d-80a5-57de29631180-tls-certs\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.205093 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.205040    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05e2863-100a-447d-80a5-57de29631180-tls-certs\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.205707 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.205668    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-dshm\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.206839 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.206794    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-home\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.207033 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.207016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-home\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.207270 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.207253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-tmp-dir\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.208140 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.207497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-model-cache\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.208431 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.208315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tqj5\" (UniqueName: \"kubernetes.io/projected/b05e2863-100a-447d-80a5-57de29631180-kube-api-access-7tqj5\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.208663 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.208060    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-tmp-dir\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.209097 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.207847    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-model-cache\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.216903 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.216841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05e2863-100a-447d-80a5-57de29631180-tls-certs\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.217369 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.217320    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-dshm\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.218847 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.218424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tqj5\" (UniqueName: \"kubernetes.io/projected/b05e2863-100a-447d-80a5-57de29631180-kube-api-access-7tqj5\") pod \"explicit-model-f324fe-kserve-678bff4bd5-d7hkl\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") " pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.360586 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.360557    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:15.636925 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.636880    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"]
Sep 17 23:21:15.639859 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:21:15.639815    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb05e2863_100a_447d_80a5_57de29631180.slice/crio-c80fb6a4fabe0ac871ab12d15368c2e283ae394c8beccb941e5b085600cb2d1a WatchSource:0}: Error finding container c80fb6a4fabe0ac871ab12d15368c2e283ae394c8beccb941e5b085600cb2d1a: Status 404 returned error can't find the container with id c80fb6a4fabe0ac871ab12d15368c2e283ae394c8beccb941e5b085600cb2d1a
Sep 17 23:21:15.863516 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.863480    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl" event={"ID":"b05e2863-100a-447d-80a5-57de29631180","Type":"ContainerStarted","Data":"a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff"}
Sep 17 23:21:15.863516 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.863521    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl" event={"ID":"b05e2863-100a-447d-80a5-57de29631180","Type":"ContainerStarted","Data":"c80fb6a4fabe0ac871ab12d15368c2e283ae394c8beccb941e5b085600cb2d1a"}
Sep 17 23:21:15.883521 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:15.883475    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl" podStartSLOduration=0.883460433 podStartE2EDuration="883.460433ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:15.882000164 +0000 UTC m=+895.886192005" watchObservedRunningTime="2026-09-17 23:21:15.883460433 +0000 UTC m=+895.887652271"
Sep 17 23:21:25.361214 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:25.361174    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:25.361596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:25.361230    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:25.373731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:25.373706    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:25.932833 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:25.932801    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:21:31.139753 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.139722    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"]
Sep 17 23:21:31.145054 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.145033    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.147750 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.147721    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8fc0d3\"/\"openshift-service-ca.crt\""
Sep 17 23:21:31.147887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.147867    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8fc0d3\"/\"kube-root-ca.crt\""
Sep 17 23:21:31.148949 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.148933    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8fc0d3\"/\"test-model-8fc0d3-kserve-self-signed-certs\""
Sep 17 23:21:31.149011 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.148989    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8fc0d3\"/\"default-dockercfg-wz2hr\""
Sep 17 23:21:31.156244 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.156221    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"]
Sep 17 23:21:31.264354 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.264323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-dshm\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.264517 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.264362    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-home\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.264517 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.264382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-model-cache\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.264517 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.264402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f7566d-3262-4401-8b85-96e49fdd6b05-tls-certs\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.264634 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.264516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24zpn\" (UniqueName: \"kubernetes.io/projected/f3f7566d-3262-4401-8b85-96e49fdd6b05-kube-api-access-24zpn\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.264634 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.264571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-tmp-dir\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366065 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366026    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-home\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366065 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-model-cache\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f7566d-3262-4401-8b85-96e49fdd6b05-tls-certs\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366197    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24zpn\" (UniqueName: \"kubernetes.io/projected/f3f7566d-3262-4401-8b85-96e49fdd6b05-kube-api-access-24zpn\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366248    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-tmp-dir\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-dshm\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366558 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366531    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-home\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366626 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366570    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-model-cache\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.366626 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.366598    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-tmp-dir\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.368523 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.368502    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-dshm\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.368668 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.368651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f7566d-3262-4401-8b85-96e49fdd6b05-tls-certs\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.374855 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.374834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24zpn\" (UniqueName: \"kubernetes.io/projected/f3f7566d-3262-4401-8b85-96e49fdd6b05-kube-api-access-24zpn\") pod \"test-model-8fc0d3-kserve-84dbd845c-w29mr\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") " pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.455244 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.455165    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:31.768818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.768792    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"]
Sep 17 23:21:31.770502 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:21:31.770471    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3f7566d_3262_4401_8b85_96e49fdd6b05.slice/crio-f7376582d0c82d10241bff9af8d1c3f93bc6921b7ccb4af12c77fe0b19f71f65 WatchSource:0}: Error finding container f7376582d0c82d10241bff9af8d1c3f93bc6921b7ccb4af12c77fe0b19f71f65: Status 404 returned error can't find the container with id f7376582d0c82d10241bff9af8d1c3f93bc6921b7ccb4af12c77fe0b19f71f65
Sep 17 23:21:31.958212 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.958173    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr" event={"ID":"f3f7566d-3262-4401-8b85-96e49fdd6b05","Type":"ContainerStarted","Data":"f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52"}
Sep 17 23:21:31.958369 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.958222    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr" event={"ID":"f3f7566d-3262-4401-8b85-96e49fdd6b05","Type":"ContainerStarted","Data":"f7376582d0c82d10241bff9af8d1c3f93bc6921b7ccb4af12c77fe0b19f71f65"}
Sep 17 23:21:31.976949 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:31.976902    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr" podStartSLOduration=0.976886867 podStartE2EDuration="976.886867ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:21:31.976038419 +0000 UTC m=+911.980230255" watchObservedRunningTime="2026-09-17 23:21:31.976886867 +0000 UTC m=+911.981078706"
Sep 17 23:21:41.455901 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:41.455862    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:41.456443 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:41.455918    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:41.468760 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:41.468732    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:42.030160 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:42.030108    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:21:50.664072 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.664033    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"]
Sep 17 23:21:50.668040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.668018    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"
Sep 17 23:21:50.670535 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.670511    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:21:50.670678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.670550    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:21:50.676519 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.676491    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"]
Sep 17 23:21:50.762774 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.762746    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a561a819-4b12-4d84-a9fa-bc20b14ec151-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl\" (UID: \"a561a819-4b12-4d84-a9fa-bc20b14ec151\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"
Sep 17 23:21:50.864440 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.864403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a561a819-4b12-4d84-a9fa-bc20b14ec151-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl\" (UID: \"a561a819-4b12-4d84-a9fa-bc20b14ec151\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"
Sep 17 23:21:50.865298 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.865273    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a561a819-4b12-4d84-a9fa-bc20b14ec151-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl\" (UID: \"a561a819-4b12-4d84-a9fa-bc20b14ec151\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"
Sep 17 23:21:50.982045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:50.981961    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"
Sep 17 23:21:51.254058 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:51.254027    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"]
Sep 17 23:21:51.255017 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:21:51.254978    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda561a819_4b12_4d84_a9fa_bc20b14ec151.slice/crio-ca289fdaa0765a25b8375b0174c3f3920f77435f338f7888d900cea4574a596f WatchSource:0}: Error finding container ca289fdaa0765a25b8375b0174c3f3920f77435f338f7888d900cea4574a596f: Status 404 returned error can't find the container with id ca289fdaa0765a25b8375b0174c3f3920f77435f338f7888d900cea4574a596f
Sep 17 23:21:52.081747 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:52.081709    2581 generic.go:356] "Generic (PLEG): container finished" podID="a561a819-4b12-4d84-a9fa-bc20b14ec151" containerID="d686b91ba25496f6df56845e60dc1ef2355ef793562e193ab2313777780aaf3e" exitCode=0
Sep 17 23:21:52.082182 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:52.081785    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl" event={"ID":"a561a819-4b12-4d84-a9fa-bc20b14ec151","Type":"ContainerDied","Data":"d686b91ba25496f6df56845e60dc1ef2355ef793562e193ab2313777780aaf3e"}
Sep 17 23:21:52.082182 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:52.081839    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl" event={"ID":"a561a819-4b12-4d84-a9fa-bc20b14ec151","Type":"ContainerStarted","Data":"ca289fdaa0765a25b8375b0174c3f3920f77435f338f7888d900cea4574a596f"}
Sep 17 23:21:53.224788 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:53.224762    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"
Sep 17 23:21:53.289063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:53.289033    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a561a819-4b12-4d84-a9fa-bc20b14ec151-maas-api-service-ca\") pod \"a561a819-4b12-4d84-a9fa-bc20b14ec151\" (UID: \"a561a819-4b12-4d84-a9fa-bc20b14ec151\") "
Sep 17 23:21:53.289400 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:53.289380    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a561a819-4b12-4d84-a9fa-bc20b14ec151-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a561a819-4b12-4d84-a9fa-bc20b14ec151" (UID: "a561a819-4b12-4d84-a9fa-bc20b14ec151"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:21:53.390599 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:53.390568    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a561a819-4b12-4d84-a9fa-bc20b14ec151-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:21:54.096544 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:54.096520    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"
Sep 17 23:21:54.096738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:54.096544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl" event={"ID":"a561a819-4b12-4d84-a9fa-bc20b14ec151","Type":"ContainerDied","Data":"ca289fdaa0765a25b8375b0174c3f3920f77435f338f7888d900cea4574a596f"}
Sep 17 23:21:54.096738 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:21:54.096590    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca289fdaa0765a25b8375b0174c3f3920f77435f338f7888d900cea4574a596f"
Sep 17 23:22:01.488024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.487987    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"]
Sep 17 23:22:01.488521 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.488248    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl" podUID="def99496-e8c2-401c-a3ec-02f5b3b3aef9" containerName="payload-processing" containerID="cri-o://d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5" gracePeriod=30
Sep 17 23:22:01.493265 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.493226    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"]
Sep 17 23:22:01.493645 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.493611    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc" podUID="6028c136-c76a-4a0d-9d08-551015e7c5c9" containerName="payload-pre-processing" containerID="cri-o://9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9" gracePeriod=30
Sep 17 23:22:01.528654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.528607    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc" podUID="6028c136-c76a-4a0d-9d08-551015e7c5c9" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.59:9004: connect: connection refused"
Sep 17 23:22:01.672423 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.672386    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"]
Sep 17 23:22:01.676178 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.676147    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-f324fe-jw4zl"]
Sep 17 23:22:01.761327 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.761301    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:22:01.792414 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.792386    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:22:01.878491 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.878454    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjg8c\" (UniqueName: \"kubernetes.io/projected/def99496-e8c2-401c-a3ec-02f5b3b3aef9-kube-api-access-mjg8c\") pod \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\" (UID: \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\") "
Sep 17 23:22:01.878491 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.878497    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def99496-e8c2-401c-a3ec-02f5b3b3aef9-plugins-config-volume\") pod \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\" (UID: \"def99496-e8c2-401c-a3ec-02f5b3b3aef9\") "
Sep 17 23:22:01.878861 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.878839    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/def99496-e8c2-401c-a3ec-02f5b3b3aef9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "def99496-e8c2-401c-a3ec-02f5b3b3aef9" (UID: "def99496-e8c2-401c-a3ec-02f5b3b3aef9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:22:01.880548 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.880525    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/def99496-e8c2-401c-a3ec-02f5b3b3aef9-kube-api-access-mjg8c" (OuterVolumeSpecName: "kube-api-access-mjg8c") pod "def99496-e8c2-401c-a3ec-02f5b3b3aef9" (UID: "def99496-e8c2-401c-a3ec-02f5b3b3aef9"). InnerVolumeSpecName "kube-api-access-mjg8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:01.980160 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.980114    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6028c136-c76a-4a0d-9d08-551015e7c5c9-plugins-config-volume\") pod \"6028c136-c76a-4a0d-9d08-551015e7c5c9\" (UID: \"6028c136-c76a-4a0d-9d08-551015e7c5c9\") "
Sep 17 23:22:01.980343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.980270    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5dgk\" (UniqueName: \"kubernetes.io/projected/6028c136-c76a-4a0d-9d08-551015e7c5c9-kube-api-access-z5dgk\") pod \"6028c136-c76a-4a0d-9d08-551015e7c5c9\" (UID: \"6028c136-c76a-4a0d-9d08-551015e7c5c9\") "
Sep 17 23:22:01.980562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.980524    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6028c136-c76a-4a0d-9d08-551015e7c5c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6028c136-c76a-4a0d-9d08-551015e7c5c9" (UID: "6028c136-c76a-4a0d-9d08-551015e7c5c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:22:01.980562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.980562    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/def99496-e8c2-401c-a3ec-02f5b3b3aef9-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:01.980713 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.980578    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjg8c\" (UniqueName: \"kubernetes.io/projected/def99496-e8c2-401c-a3ec-02f5b3b3aef9-kube-api-access-mjg8c\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:01.982394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.982370    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6028c136-c76a-4a0d-9d08-551015e7c5c9-kube-api-access-z5dgk" (OuterVolumeSpecName: "kube-api-access-z5dgk") pod "6028c136-c76a-4a0d-9d08-551015e7c5c9" (UID: "6028c136-c76a-4a0d-9d08-551015e7c5c9"). InnerVolumeSpecName "kube-api-access-z5dgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:01.994275 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:01.994215    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a561a819-4b12-4d84-a9fa-bc20b14ec151" path="/var/lib/kubelet/pods/a561a819-4b12-4d84-a9fa-bc20b14ec151/volumes"
Sep 17 23:22:02.081612 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.081584    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5dgk\" (UniqueName: \"kubernetes.io/projected/6028c136-c76a-4a0d-9d08-551015e7c5c9-kube-api-access-z5dgk\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:02.081612 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.081611    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6028c136-c76a-4a0d-9d08-551015e7c5c9-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:02.149104 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.149065    2581 generic.go:356] "Generic (PLEG): container finished" podID="6028c136-c76a-4a0d-9d08-551015e7c5c9" containerID="9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9" exitCode=0
Sep 17 23:22:02.149313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.149119    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"
Sep 17 23:22:02.149313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.149156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc" event={"ID":"6028c136-c76a-4a0d-9d08-551015e7c5c9","Type":"ContainerDied","Data":"9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9"}
Sep 17 23:22:02.149313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.149220    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc" event={"ID":"6028c136-c76a-4a0d-9d08-551015e7c5c9","Type":"ContainerDied","Data":"5434afd6ad2c6bc2c46bbfd4bbe61e34b26ba96a17fbc57d991adced76d6287d"}
Sep 17 23:22:02.149313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.149250    2581 scope.go:117] "RemoveContainer" containerID="9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9"
Sep 17 23:22:02.150848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.150823    2581 generic.go:356] "Generic (PLEG): container finished" podID="def99496-e8c2-401c-a3ec-02f5b3b3aef9" containerID="d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5" exitCode=0
Sep 17 23:22:02.150930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.150865    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"
Sep 17 23:22:02.150930 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.150912    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl" event={"ID":"def99496-e8c2-401c-a3ec-02f5b3b3aef9","Type":"ContainerDied","Data":"d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5"}
Sep 17 23:22:02.151036 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.150954    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl" event={"ID":"def99496-e8c2-401c-a3ec-02f5b3b3aef9","Type":"ContainerDied","Data":"3dfd1993a8627fbd1c118415b108ca5f2299746e9ab54646c476c444e3bc1436"}
Sep 17 23:22:02.159345 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.159330    2581 scope.go:117] "RemoveContainer" containerID="9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9"
Sep 17 23:22:02.159621 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:22:02.159602    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9\": container with ID starting with 9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9 not found: ID does not exist" containerID="9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9"
Sep 17 23:22:02.159659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.159629    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9"} err="failed to get container status \"9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9\": rpc error: code = NotFound desc = could not find container \"9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9\": container with ID starting with 9a152747439f06310a8a515024a4dcf21430ed721ff6f2019d195597b6f236e9 not found: ID does not exist"
Sep 17 23:22:02.159659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.159646    2581 scope.go:117] "RemoveContainer" containerID="d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5"
Sep 17 23:22:02.168488 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.168456    2581 scope.go:117] "RemoveContainer" containerID="d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5"
Sep 17 23:22:02.168762 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:22:02.168744    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5\": container with ID starting with d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5 not found: ID does not exist" containerID="d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5"
Sep 17 23:22:02.168831 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.168774    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5"} err="failed to get container status \"d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5\": rpc error: code = NotFound desc = could not find container \"d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5\": container with ID starting with d33c5eae6f3feab9c22665f9e96d467b5d6d19eaa415d835bdb7168d6e5700f5 not found: ID does not exist"
Sep 17 23:22:02.172365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.172344    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"]
Sep 17 23:22:02.176576 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.176556    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-f324fe-5bd7c9f4f4-mttpc"]
Sep 17 23:22:02.194329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.194306    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"]
Sep 17 23:22:02.199748 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:02.199726    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f324fe-685449dd65-vr8kl"]
Sep 17 23:22:03.994035 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:03.994001    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6028c136-c76a-4a0d-9d08-551015e7c5c9" path="/var/lib/kubelet/pods/6028c136-c76a-4a0d-9d08-551015e7c5c9/volumes"
Sep 17 23:22:03.994433 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:03.994411    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="def99496-e8c2-401c-a3ec-02f5b3b3aef9" path="/var/lib/kubelet/pods/def99496-e8c2-401c-a3ec-02f5b3b3aef9/volumes"
Sep 17 23:22:06.999476 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:06.999438    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"]
Sep 17 23:22:10.339699 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.339664    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"]
Sep 17 23:22:10.340090 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340072    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="def99496-e8c2-401c-a3ec-02f5b3b3aef9" containerName="payload-processing"
Sep 17 23:22:10.340090 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340084    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="def99496-e8c2-401c-a3ec-02f5b3b3aef9" containerName="payload-processing"
Sep 17 23:22:10.340200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340096    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6028c136-c76a-4a0d-9d08-551015e7c5c9" containerName="payload-pre-processing"
Sep 17 23:22:10.340200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340102    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6028c136-c76a-4a0d-9d08-551015e7c5c9" containerName="payload-pre-processing"
Sep 17 23:22:10.340200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340109    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a561a819-4b12-4d84-a9fa-bc20b14ec151" containerName="revoke-keys"
Sep 17 23:22:10.340200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340116    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a561a819-4b12-4d84-a9fa-bc20b14ec151" containerName="revoke-keys"
Sep 17 23:22:10.340200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340185    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6028c136-c76a-4a0d-9d08-551015e7c5c9" containerName="payload-pre-processing"
Sep 17 23:22:10.340200 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340195    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="def99496-e8c2-401c-a3ec-02f5b3b3aef9" containerName="payload-processing"
Sep 17 23:22:10.340382 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.340203    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a561a819-4b12-4d84-a9fa-bc20b14ec151" containerName="revoke-keys"
Sep 17 23:22:10.343286 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.343270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.348069 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.348049    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-7ab633\"/\"e2e-conflict-a-21551ab9-kserve-self-signed-certs\""
Sep 17 23:22:10.359268 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.359248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-home\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.359391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.359278    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-model-cache\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.359391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.359368    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5jjj\" (UniqueName: \"kubernetes.io/projected/b41c8292-07b1-45d8-aa34-26130de9a959-kube-api-access-t5jjj\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.359547 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.359395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-tmp-dir\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.359547 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.359471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-dshm\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.359663 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.359622    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41c8292-07b1-45d8-aa34-26130de9a959-tls-certs\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.363752 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.363729    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"]
Sep 17 23:22:10.460531 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.460497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-home\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.460531 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.460534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-model-cache\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.460761 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.460559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5jjj\" (UniqueName: \"kubernetes.io/projected/b41c8292-07b1-45d8-aa34-26130de9a959-kube-api-access-t5jjj\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.460761 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.460637    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-tmp-dir\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.460761 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.460699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-dshm\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.460931 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.460853    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41c8292-07b1-45d8-aa34-26130de9a959-tls-certs\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.460931 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.460924    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-home\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.461042 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.460968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-model-cache\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.461042 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.461008    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-tmp-dir\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.462979 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.462955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-dshm\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.463527 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.463508    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41c8292-07b1-45d8-aa34-26130de9a959-tls-certs\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.468891 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.468871    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5jjj\" (UniqueName: \"kubernetes.io/projected/b41c8292-07b1-45d8-aa34-26130de9a959-kube-api-access-t5jjj\") pod \"e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") " pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.652365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.652327    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:10.941598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:10.941559    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"]
Sep 17 23:22:10.942080 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:22:10.942054    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb41c8292_07b1_45d8_aa34_26130de9a959.slice/crio-87561251ca2172b907d92d7875c42c30877ab3c4a21b1fc77ea54215b8fff0ba WatchSource:0}: Error finding container 87561251ca2172b907d92d7875c42c30877ab3c4a21b1fc77ea54215b8fff0ba: Status 404 returned error can't find the container with id 87561251ca2172b907d92d7875c42c30877ab3c4a21b1fc77ea54215b8fff0ba
Sep 17 23:22:11.216159 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.216060    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs" event={"ID":"b41c8292-07b1-45d8-aa34-26130de9a959","Type":"ContainerStarted","Data":"d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41"}
Sep 17 23:22:11.216159 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.216103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs" event={"ID":"b41c8292-07b1-45d8-aa34-26130de9a959","Type":"ContainerStarted","Data":"87561251ca2172b907d92d7875c42c30877ab3c4a21b1fc77ea54215b8fff0ba"}
Sep 17 23:22:11.245780 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.245731    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs" podStartSLOduration=1.245716129 podStartE2EDuration="1.245716129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:11.242468066 +0000 UTC m=+951.246659905" watchObservedRunningTime="2026-09-17 23:22:11.245716129 +0000 UTC m=+951.249907970"
Sep 17 23:22:11.439480 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.439444    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"]
Sep 17 23:22:11.449152 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.443583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:11.449152 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.446785    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-700887b8\""
Sep 17 23:22:11.449152 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.447171    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-700887b8-dockercfg-j9r2n\""
Sep 17 23:22:11.460315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.460290    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"]
Sep 17 23:22:11.471870 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.471781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz7ld\" (UniqueName: \"kubernetes.io/projected/3d6bbc55-2fd9-45d7-98f0-928047f17967-kube-api-access-wz7ld\") pod \"payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb\" (UID: \"3d6bbc55-2fd9-45d7-98f0-928047f17967\") " pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:11.471870 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.471855    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d6bbc55-2fd9-45d7-98f0-928047f17967-plugins-config-volume\") pod \"payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb\" (UID: \"3d6bbc55-2fd9-45d7-98f0-928047f17967\") " pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:11.574148 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.573710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz7ld\" (UniqueName: \"kubernetes.io/projected/3d6bbc55-2fd9-45d7-98f0-928047f17967-kube-api-access-wz7ld\") pod \"payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb\" (UID: \"3d6bbc55-2fd9-45d7-98f0-928047f17967\") " pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:11.574148 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.573792    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d6bbc55-2fd9-45d7-98f0-928047f17967-plugins-config-volume\") pod \"payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb\" (UID: \"3d6bbc55-2fd9-45d7-98f0-928047f17967\") " pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:11.574935 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.574826    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d6bbc55-2fd9-45d7-98f0-928047f17967-plugins-config-volume\") pod \"payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb\" (UID: \"3d6bbc55-2fd9-45d7-98f0-928047f17967\") " pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:11.589520 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.589491    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz7ld\" (UniqueName: \"kubernetes.io/projected/3d6bbc55-2fd9-45d7-98f0-928047f17967-kube-api-access-wz7ld\") pod \"payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb\" (UID: \"3d6bbc55-2fd9-45d7-98f0-928047f17967\") " pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:11.756307 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:11.756225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:12.034041 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:12.034011    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"]
Sep 17 23:22:12.035068 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:22:12.035040    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d6bbc55_2fd9_45d7_98f0_928047f17967.slice/crio-9de69a74b0b900ade33be19534df1d0d86a5949e103e37671740bed0aa816345 WatchSource:0}: Error finding container 9de69a74b0b900ade33be19534df1d0d86a5949e103e37671740bed0aa816345: Status 404 returned error can't find the container with id 9de69a74b0b900ade33be19534df1d0d86a5949e103e37671740bed0aa816345
Sep 17 23:22:12.226345 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:12.226303    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb" event={"ID":"3d6bbc55-2fd9-45d7-98f0-928047f17967","Type":"ContainerStarted","Data":"906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe"}
Sep 17 23:22:12.226507 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:12.226364    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:12.226507 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:12.226388    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb" event={"ID":"3d6bbc55-2fd9-45d7-98f0-928047f17967","Type":"ContainerStarted","Data":"9de69a74b0b900ade33be19534df1d0d86a5949e103e37671740bed0aa816345"}
Sep 17 23:22:12.248866 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:12.248803    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb" podStartSLOduration=1.248782201 podStartE2EDuration="1.248782201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:12.244836237 +0000 UTC m=+952.249028071" watchObservedRunningTime="2026-09-17 23:22:12.248782201 +0000 UTC m=+952.252974042"
Sep 17 23:22:20.653242 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:20.653200    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:20.653642 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:20.653255    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:20.665510 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:20.665487    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:21.290300 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:21.290269    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:22:22.019210 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.019169    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl" podUID="b05e2863-100a-447d-80a5-57de29631180" containerName="main" containerID="cri-o://a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff" gracePeriod=45
Sep 17 23:22:22.272543 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.272486    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:22:22.286675 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.286648    2581 generic.go:356] "Generic (PLEG): container finished" podID="b05e2863-100a-447d-80a5-57de29631180" containerID="a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff" exitCode=0
Sep 17 23:22:22.286822 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.286686    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"
Sep 17 23:22:22.286822 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.286732    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl" event={"ID":"b05e2863-100a-447d-80a5-57de29631180","Type":"ContainerDied","Data":"a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff"}
Sep 17 23:22:22.286822 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.286781    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl" event={"ID":"b05e2863-100a-447d-80a5-57de29631180","Type":"ContainerDied","Data":"c80fb6a4fabe0ac871ab12d15368c2e283ae394c8beccb941e5b085600cb2d1a"}
Sep 17 23:22:22.286822 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.286807    2581 scope.go:117] "RemoveContainer" containerID="a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff"
Sep 17 23:22:22.297647 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.297626    2581 scope.go:117] "RemoveContainer" containerID="a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff"
Sep 17 23:22:22.297930 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:22:22.297910    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff\": container with ID starting with a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff not found: ID does not exist" containerID="a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff"
Sep 17 23:22:22.298023 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.297937    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff"} err="failed to get container status \"a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff\": rpc error: code = NotFound desc = could not find container \"a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff\": container with ID starting with a826f2bb4a58374d11c2c284eb3c7a89aa91efaacda4e11f7670e4d08da20bff not found: ID does not exist"
Sep 17 23:22:22.383311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383279    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-dshm\") pod \"b05e2863-100a-447d-80a5-57de29631180\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") "
Sep 17 23:22:22.383311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383314    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05e2863-100a-447d-80a5-57de29631180-tls-certs\") pod \"b05e2863-100a-447d-80a5-57de29631180\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") "
Sep 17 23:22:22.383552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383351    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-model-cache\") pod \"b05e2863-100a-447d-80a5-57de29631180\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") "
Sep 17 23:22:22.383552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383367    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-home\") pod \"b05e2863-100a-447d-80a5-57de29631180\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") "
Sep 17 23:22:22.383552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383439    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tqj5\" (UniqueName: \"kubernetes.io/projected/b05e2863-100a-447d-80a5-57de29631180-kube-api-access-7tqj5\") pod \"b05e2863-100a-447d-80a5-57de29631180\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") "
Sep 17 23:22:22.383552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383481    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-tmp-dir\") pod \"b05e2863-100a-447d-80a5-57de29631180\" (UID: \"b05e2863-100a-447d-80a5-57de29631180\") "
Sep 17 23:22:22.383765 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383610    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-model-cache" (OuterVolumeSpecName: "model-cache") pod "b05e2863-100a-447d-80a5-57de29631180" (UID: "b05e2863-100a-447d-80a5-57de29631180"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:22.383765 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383694    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-home" (OuterVolumeSpecName: "home") pod "b05e2863-100a-447d-80a5-57de29631180" (UID: "b05e2863-100a-447d-80a5-57de29631180"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:22.383859 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383761    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b05e2863-100a-447d-80a5-57de29631180" (UID: "b05e2863-100a-447d-80a5-57de29631180"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:22.383978 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383959    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:22.384042 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383983    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:22.384042 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.383997    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:22.385684 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.385658    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-dshm" (OuterVolumeSpecName: "dshm") pod "b05e2863-100a-447d-80a5-57de29631180" (UID: "b05e2863-100a-447d-80a5-57de29631180"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:22:22.385784 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.385724    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b05e2863-100a-447d-80a5-57de29631180-kube-api-access-7tqj5" (OuterVolumeSpecName: "kube-api-access-7tqj5") pod "b05e2863-100a-447d-80a5-57de29631180" (UID: "b05e2863-100a-447d-80a5-57de29631180"). InnerVolumeSpecName "kube-api-access-7tqj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:22.385838 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.385801    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b05e2863-100a-447d-80a5-57de29631180-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b05e2863-100a-447d-80a5-57de29631180" (UID: "b05e2863-100a-447d-80a5-57de29631180"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:22:22.484924 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.484894    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b05e2863-100a-447d-80a5-57de29631180-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:22.484924 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.484922    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b05e2863-100a-447d-80a5-57de29631180-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:22.485090 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.484933    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tqj5\" (UniqueName: \"kubernetes.io/projected/b05e2863-100a-447d-80a5-57de29631180-kube-api-access-7tqj5\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:22.609648 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.609619    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"]
Sep 17 23:22:22.613075 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:22.613050    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-f324fe/explicit-model-f324fe-kserve-678bff4bd5-d7hkl"]
Sep 17 23:22:23.235144 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:23.235089    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:23.994350 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:23.994310    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b05e2863-100a-447d-80a5-57de29631180" path="/var/lib/kubelet/pods/b05e2863-100a-447d-80a5-57de29631180/volumes"
Sep 17 23:22:43.408073 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.407994    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"]
Sep 17 23:22:43.408528 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.408448    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b05e2863-100a-447d-80a5-57de29631180" containerName="main"
Sep 17 23:22:43.408528 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.408461    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b05e2863-100a-447d-80a5-57de29631180" containerName="main"
Sep 17 23:22:43.408528 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.408525    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b05e2863-100a-447d-80a5-57de29631180" containerName="main"
Sep 17 23:22:43.411652 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.411629    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"
Sep 17 23:22:43.414357 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.414337    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:22:43.414498 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.414479    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:22:43.420438 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.420206    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"]
Sep 17 23:22:43.494282 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.494240    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5e61-22e5-4e44-a51c-5a1b91435630-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-700887b8-ph27j\" (UID: \"97bc5e61-22e5-4e44-a51c-5a1b91435630\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"
Sep 17 23:22:43.595082 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.595050    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5e61-22e5-4e44-a51c-5a1b91435630-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-700887b8-ph27j\" (UID: \"97bc5e61-22e5-4e44-a51c-5a1b91435630\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"
Sep 17 23:22:43.595723 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.595702    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5e61-22e5-4e44-a51c-5a1b91435630-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-700887b8-ph27j\" (UID: \"97bc5e61-22e5-4e44-a51c-5a1b91435630\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"
Sep 17 23:22:43.724365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:43.724265    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"
Sep 17 23:22:44.000152 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:44.000109    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"]
Sep 17 23:22:44.000844 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:22:44.000814    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bc5e61_22e5_4e44_a51c_5a1b91435630.slice/crio-29617d26af079bf07d849d5aae9ff7d5379b98ff23bf9d5d735cd33dc076d59b WatchSource:0}: Error finding container 29617d26af079bf07d849d5aae9ff7d5379b98ff23bf9d5d735cd33dc076d59b: Status 404 returned error can't find the container with id 29617d26af079bf07d849d5aae9ff7d5379b98ff23bf9d5d735cd33dc076d59b
Sep 17 23:22:44.428382 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:44.428347    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j" event={"ID":"97bc5e61-22e5-4e44-a51c-5a1b91435630","Type":"ContainerStarted","Data":"cf46a71dc1537c2dbd6989c730bdacf1616c9530e3be067a0bf03b9c986adbb1"}
Sep 17 23:22:44.428763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:44.428401    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j" event={"ID":"97bc5e61-22e5-4e44-a51c-5a1b91435630","Type":"ContainerStarted","Data":"29617d26af079bf07d849d5aae9ff7d5379b98ff23bf9d5d735cd33dc076d59b"}
Sep 17 23:22:44.444239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:44.444182    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j" podStartSLOduration=1.444163633 podStartE2EDuration="1.444163633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:22:44.443955993 +0000 UTC m=+984.448147833" watchObservedRunningTime="2026-09-17 23:22:44.444163633 +0000 UTC m=+984.448355475"
Sep 17 23:22:45.436120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:45.436084    2581 generic.go:356] "Generic (PLEG): container finished" podID="97bc5e61-22e5-4e44-a51c-5a1b91435630" containerID="cf46a71dc1537c2dbd6989c730bdacf1616c9530e3be067a0bf03b9c986adbb1" exitCode=0
Sep 17 23:22:45.436541 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:45.436152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j" event={"ID":"97bc5e61-22e5-4e44-a51c-5a1b91435630","Type":"ContainerDied","Data":"cf46a71dc1537c2dbd6989c730bdacf1616c9530e3be067a0bf03b9c986adbb1"}
Sep 17 23:22:46.582091 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:46.582066    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"
Sep 17 23:22:46.625071 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:46.625041    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5e61-22e5-4e44-a51c-5a1b91435630-maas-api-service-ca\") pod \"97bc5e61-22e5-4e44-a51c-5a1b91435630\" (UID: \"97bc5e61-22e5-4e44-a51c-5a1b91435630\") "
Sep 17 23:22:46.625433 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:46.625408    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97bc5e61-22e5-4e44-a51c-5a1b91435630-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97bc5e61-22e5-4e44-a51c-5a1b91435630" (UID: "97bc5e61-22e5-4e44-a51c-5a1b91435630"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:22:46.726193 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:46.726084    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97bc5e61-22e5-4e44-a51c-5a1b91435630-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:47.449326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:47.449295    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j" event={"ID":"97bc5e61-22e5-4e44-a51c-5a1b91435630","Type":"ContainerDied","Data":"29617d26af079bf07d849d5aae9ff7d5379b98ff23bf9d5d735cd33dc076d59b"}
Sep 17 23:22:47.449326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:47.449315    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"
Sep 17 23:22:47.449326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:47.449343    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29617d26af079bf07d849d5aae9ff7d5379b98ff23bf9d5d735cd33dc076d59b"
Sep 17 23:22:53.932838 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:53.932802    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"]
Sep 17 23:22:53.933370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:53.933148    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb" podUID="3d6bbc55-2fd9-45d7-98f0-928047f17967" containerName="payload-processing" containerID="cri-o://906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe" gracePeriod=30
Sep 17 23:22:54.090317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.090279    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"]
Sep 17 23:22:54.092805 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.092780    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-700887b8-ph27j"]
Sep 17 23:22:54.215742 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.215699    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:54.308114 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.308077    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d6bbc55-2fd9-45d7-98f0-928047f17967-plugins-config-volume\") pod \"3d6bbc55-2fd9-45d7-98f0-928047f17967\" (UID: \"3d6bbc55-2fd9-45d7-98f0-928047f17967\") "
Sep 17 23:22:54.308300 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.308225    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wz7ld\" (UniqueName: \"kubernetes.io/projected/3d6bbc55-2fd9-45d7-98f0-928047f17967-kube-api-access-wz7ld\") pod \"3d6bbc55-2fd9-45d7-98f0-928047f17967\" (UID: \"3d6bbc55-2fd9-45d7-98f0-928047f17967\") "
Sep 17 23:22:54.308462 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.308437    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d6bbc55-2fd9-45d7-98f0-928047f17967-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3d6bbc55-2fd9-45d7-98f0-928047f17967" (UID: "3d6bbc55-2fd9-45d7-98f0-928047f17967"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:22:54.308569 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.308555    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d6bbc55-2fd9-45d7-98f0-928047f17967-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:54.310286 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.310267    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d6bbc55-2fd9-45d7-98f0-928047f17967-kube-api-access-wz7ld" (OuterVolumeSpecName: "kube-api-access-wz7ld") pod "3d6bbc55-2fd9-45d7-98f0-928047f17967" (UID: "3d6bbc55-2fd9-45d7-98f0-928047f17967"). InnerVolumeSpecName "kube-api-access-wz7ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:22:54.409509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.409475    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wz7ld\" (UniqueName: \"kubernetes.io/projected/3d6bbc55-2fd9-45d7-98f0-928047f17967-kube-api-access-wz7ld\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:22:54.492579 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.492490    2581 generic.go:356] "Generic (PLEG): container finished" podID="3d6bbc55-2fd9-45d7-98f0-928047f17967" containerID="906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe" exitCode=0
Sep 17 23:22:54.492579 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.492542    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"
Sep 17 23:22:54.492579 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.492567    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb" event={"ID":"3d6bbc55-2fd9-45d7-98f0-928047f17967","Type":"ContainerDied","Data":"906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe"}
Sep 17 23:22:54.492819 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.492620    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb" event={"ID":"3d6bbc55-2fd9-45d7-98f0-928047f17967","Type":"ContainerDied","Data":"9de69a74b0b900ade33be19534df1d0d86a5949e103e37671740bed0aa816345"}
Sep 17 23:22:54.492819 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.492645    2581 scope.go:117] "RemoveContainer" containerID="906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe"
Sep 17 23:22:54.502415 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.502399    2581 scope.go:117] "RemoveContainer" containerID="906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe"
Sep 17 23:22:54.502700 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:22:54.502680    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe\": container with ID starting with 906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe not found: ID does not exist" containerID="906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe"
Sep 17 23:22:54.502807 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.502705    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe"} err="failed to get container status \"906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe\": rpc error: code = NotFound desc = could not find container \"906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe\": container with ID starting with 906fdde0fcae4baa5d0999c3108368a481b321ab779bd25e942afc70be286dfe not found: ID does not exist"
Sep 17 23:22:54.517058 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.517026    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"]
Sep 17 23:22:54.521167 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:54.521117    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-700887b8-6f44bd6c7c-jtzjb"]
Sep 17 23:22:55.997378 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:55.997342    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d6bbc55-2fd9-45d7-98f0-928047f17967" path="/var/lib/kubelet/pods/3d6bbc55-2fd9-45d7-98f0-928047f17967/volumes"
Sep 17 23:22:55.998559 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:22:55.998526    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97bc5e61-22e5-4e44-a51c-5a1b91435630" path="/var/lib/kubelet/pods/97bc5e61-22e5-4e44-a51c-5a1b91435630/volumes"
Sep 17 23:23:04.041751 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.041721    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"]
Sep 17 23:23:04.042320 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.042302    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97bc5e61-22e5-4e44-a51c-5a1b91435630" containerName="revoke-keys"
Sep 17 23:23:04.042320 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.042321    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="97bc5e61-22e5-4e44-a51c-5a1b91435630" containerName="revoke-keys"
Sep 17 23:23:04.042451 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.042334    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d6bbc55-2fd9-45d7-98f0-928047f17967" containerName="payload-processing"
Sep 17 23:23:04.042451 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.042340    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d6bbc55-2fd9-45d7-98f0-928047f17967" containerName="payload-processing"
Sep 17 23:23:04.042451 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.042408    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d6bbc55-2fd9-45d7-98f0-928047f17967" containerName="payload-processing"
Sep 17 23:23:04.042451 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.042417    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="97bc5e61-22e5-4e44-a51c-5a1b91435630" containerName="revoke-keys"
Sep 17 23:23:04.047106 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.047089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.051223 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.051200    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-b6f859\"/\"openshift-service-ca.crt\""
Sep 17 23:23:04.051351 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.051224    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-b6f859\"/\"kube-root-ca.crt\""
Sep 17 23:23:04.051351 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.051206    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-b6f859\"/\"default-dockercfg-z29kk\""
Sep 17 23:23:04.051351 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.051203    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-b6f859\"/\"noten-model-b6f859-kserve-self-signed-certs\""
Sep 17 23:23:04.055800 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.055777    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"]
Sep 17 23:23:04.211744 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.211715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-dshm\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.211908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.211773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/235942d0-b7cd-43a8-aa83-780ed533779a-tls-certs\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.211908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.211826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-model-cache\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.211908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.211905    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwzwq\" (UniqueName: \"kubernetes.io/projected/235942d0-b7cd-43a8-aa83-780ed533779a-kube-api-access-qwzwq\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.212029 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.211924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-home\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.212029 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.211945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-tmp-dir\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.313542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.313442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/235942d0-b7cd-43a8-aa83-780ed533779a-tls-certs\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.313542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.313507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-model-cache\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.313767 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.313582    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwzwq\" (UniqueName: \"kubernetes.io/projected/235942d0-b7cd-43a8-aa83-780ed533779a-kube-api-access-qwzwq\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.313767 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.313611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-home\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.313767 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.313639    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-tmp-dir\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.313767 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.313718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-dshm\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.314243 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.314208    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-model-cache\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.314367 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.314278    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-home\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.314367 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.314331    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-tmp-dir\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.316103 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.316079    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-dshm\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.316480 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.316463    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/235942d0-b7cd-43a8-aa83-780ed533779a-tls-certs\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.322894 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.322870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwzwq\" (UniqueName: \"kubernetes.io/projected/235942d0-b7cd-43a8-aa83-780ed533779a-kube-api-access-qwzwq\") pod \"noten-model-b6f859-kserve-68f9dcb88c-lct64\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") " pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.357582 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.357554    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:04.634753 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:04.634722    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"]
Sep 17 23:23:04.635948 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:23:04.635916    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod235942d0_b7cd_43a8_aa83_780ed533779a.slice/crio-be708661040a013765957d9da460e4d8fa0ba488d42b9769f2c3a4054f8b28a3 WatchSource:0}: Error finding container be708661040a013765957d9da460e4d8fa0ba488d42b9769f2c3a4054f8b28a3: Status 404 returned error can't find the container with id be708661040a013765957d9da460e4d8fa0ba488d42b9769f2c3a4054f8b28a3
Sep 17 23:23:05.563477 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:05.563438    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64" event={"ID":"235942d0-b7cd-43a8-aa83-780ed533779a","Type":"ContainerStarted","Data":"bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e"}
Sep 17 23:23:05.563865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:05.563489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64" event={"ID":"235942d0-b7cd-43a8-aa83-780ed533779a","Type":"ContainerStarted","Data":"be708661040a013765957d9da460e4d8fa0ba488d42b9769f2c3a4054f8b28a3"}
Sep 17 23:23:05.602780 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:05.602734    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64" podStartSLOduration=1.6027199589999999 podStartE2EDuration="1.602719959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:23:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:23:05.599820234 +0000 UTC m=+1005.604012074" watchObservedRunningTime="2026-09-17 23:23:05.602719959 +0000 UTC m=+1005.606911797"
Sep 17 23:23:14.357982 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:14.357942    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:14.358583 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:14.357995    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:14.370661 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:14.370637    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:14.629681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:14.629655    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:21.872618 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:21.872580    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"]
Sep 17 23:23:21.876251 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:21.876233    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:21.879017 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:21.878992    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-09c1c1\""
Sep 17 23:23:21.879166 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:21.879098    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-09c1c1-dockercfg-wdgnz\""
Sep 17 23:23:21.887709 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:21.887684    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"]
Sep 17 23:23:21.913440 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:21.913413    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"]
Sep 17 23:23:21.916989 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:21.916970    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:21.933554 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:21.933521    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"]
Sep 17 23:23:22.009024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.007505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg\" (UID: \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:22.009024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.007552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdpgx\" (UniqueName: \"kubernetes.io/projected/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-kube-api-access-kdpgx\") pod \"payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg\" (UID: \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:22.009024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.007608    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d229854a-a04b-42fc-ace8-a5ac99979da8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv\" (UID: \"d229854a-a04b-42fc-ace8-a5ac99979da8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:22.009024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.007731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4ltv\" (UniqueName: \"kubernetes.io/projected/d229854a-a04b-42fc-ace8-a5ac99979da8-kube-api-access-n4ltv\") pod \"payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv\" (UID: \"d229854a-a04b-42fc-ace8-a5ac99979da8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:22.108775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.108733    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg\" (UID: \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:22.108968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.108782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdpgx\" (UniqueName: \"kubernetes.io/projected/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-kube-api-access-kdpgx\") pod \"payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg\" (UID: \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:22.108968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.108828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d229854a-a04b-42fc-ace8-a5ac99979da8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv\" (UID: \"d229854a-a04b-42fc-ace8-a5ac99979da8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:22.108968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.108915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4ltv\" (UniqueName: \"kubernetes.io/projected/d229854a-a04b-42fc-ace8-a5ac99979da8-kube-api-access-n4ltv\") pod \"payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv\" (UID: \"d229854a-a04b-42fc-ace8-a5ac99979da8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:22.109586 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.109556    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg\" (UID: \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:22.109734 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.109710    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d229854a-a04b-42fc-ace8-a5ac99979da8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv\" (UID: \"d229854a-a04b-42fc-ace8-a5ac99979da8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:22.117473 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.117444    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4ltv\" (UniqueName: \"kubernetes.io/projected/d229854a-a04b-42fc-ace8-a5ac99979da8-kube-api-access-n4ltv\") pod \"payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv\" (UID: \"d229854a-a04b-42fc-ace8-a5ac99979da8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:22.117564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.117516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdpgx\" (UniqueName: \"kubernetes.io/projected/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-kube-api-access-kdpgx\") pod \"payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg\" (UID: \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:22.187741 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.187712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:22.235515 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.235478    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:22.465555 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.465520    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"]
Sep 17 23:23:22.466032 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:23:22.466006    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b06b80_0e03_41f5_b00b_a131ae8e98cb.slice/crio-2c604770ca43ccf4eebce65cac10643c9f30cdf5a29c6180621245621693f8e6 WatchSource:0}: Error finding container 2c604770ca43ccf4eebce65cac10643c9f30cdf5a29c6180621245621693f8e6: Status 404 returned error can't find the container with id 2c604770ca43ccf4eebce65cac10643c9f30cdf5a29c6180621245621693f8e6
Sep 17 23:23:22.536411 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.536385    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"]
Sep 17 23:23:22.537080 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:23:22.537050    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd229854a_a04b_42fc_ace8_a5ac99979da8.slice/crio-e8e099ef84a6e4cf79adb62840bc5b97a1c9542d9f570adae5f72188657bde23 WatchSource:0}: Error finding container e8e099ef84a6e4cf79adb62840bc5b97a1c9542d9f570adae5f72188657bde23: Status 404 returned error can't find the container with id e8e099ef84a6e4cf79adb62840bc5b97a1c9542d9f570adae5f72188657bde23
Sep 17 23:23:22.670927 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.670887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg" event={"ID":"a4b06b80-0e03-41f5-b00b-a131ae8e98cb","Type":"ContainerStarted","Data":"510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603"}
Sep 17 23:23:22.671137 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.670955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg" event={"ID":"a4b06b80-0e03-41f5-b00b-a131ae8e98cb","Type":"ContainerStarted","Data":"2c604770ca43ccf4eebce65cac10643c9f30cdf5a29c6180621245621693f8e6"}
Sep 17 23:23:22.671241 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.671214    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:22.672965 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.672938    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv" event={"ID":"d229854a-a04b-42fc-ace8-a5ac99979da8","Type":"ContainerStarted","Data":"bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b"}
Sep 17 23:23:22.673065 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.672987    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv" event={"ID":"d229854a-a04b-42fc-ace8-a5ac99979da8","Type":"ContainerStarted","Data":"e8e099ef84a6e4cf79adb62840bc5b97a1c9542d9f570adae5f72188657bde23"}
Sep 17 23:23:22.673065 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.673020    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:22.690174 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.690114    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg" podStartSLOduration=1.6900983950000001 podStartE2EDuration="1.690098395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:23:22.687175056 +0000 UTC m=+1022.691366887" watchObservedRunningTime="2026-09-17 23:23:22.690098395 +0000 UTC m=+1022.694290233"
Sep 17 23:23:22.707990 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:22.707936    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv" podStartSLOduration=1.707913955 podStartE2EDuration="1.707913955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:23:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:23:22.703786258 +0000 UTC m=+1022.707978097" watchObservedRunningTime="2026-09-17 23:23:22.707913955 +0000 UTC m=+1022.712105800"
Sep 17 23:23:33.681322 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:33.681288    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:23:33.683410 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:33.681621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:23:43.588413 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:43.588373    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"]
Sep 17 23:23:56.761758 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.761715    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"]
Sep 17 23:23:56.765898 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.765859    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:56.769281 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.769259    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-55130c\""
Sep 17 23:23:56.770428 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.770407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-55130c-dockercfg-fq566\""
Sep 17 23:23:56.785837 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.785813    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"]
Sep 17 23:23:56.859276 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.859231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wr92\" (UniqueName: \"kubernetes.io/projected/f6e4f647-6569-4236-aabb-ebd6aab162f2-kube-api-access-4wr92\") pod \"payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd\" (UID: \"f6e4f647-6569-4236-aabb-ebd6aab162f2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:56.859481 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.859366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e4f647-6569-4236-aabb-ebd6aab162f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd\" (UID: \"f6e4f647-6569-4236-aabb-ebd6aab162f2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:56.960361 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.960329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wr92\" (UniqueName: \"kubernetes.io/projected/f6e4f647-6569-4236-aabb-ebd6aab162f2-kube-api-access-4wr92\") pod \"payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd\" (UID: \"f6e4f647-6569-4236-aabb-ebd6aab162f2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:56.960546 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.960428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e4f647-6569-4236-aabb-ebd6aab162f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd\" (UID: \"f6e4f647-6569-4236-aabb-ebd6aab162f2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:56.961166 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.961113    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e4f647-6569-4236-aabb-ebd6aab162f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd\" (UID: \"f6e4f647-6569-4236-aabb-ebd6aab162f2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:56.968900 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:56.968873    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wr92\" (UniqueName: \"kubernetes.io/projected/f6e4f647-6569-4236-aabb-ebd6aab162f2-kube-api-access-4wr92\") pod \"payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd\" (UID: \"f6e4f647-6569-4236-aabb-ebd6aab162f2\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:57.076285 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:57.076202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:57.351615 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:57.351585    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"]
Sep 17 23:23:57.353135 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:23:57.353103    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e4f647_6569_4236_aabb_ebd6aab162f2.slice/crio-294f0519043981657168711190701420bceca0c21511219c3a8eadfaa6e6edb0 WatchSource:0}: Error finding container 294f0519043981657168711190701420bceca0c21511219c3a8eadfaa6e6edb0: Status 404 returned error can't find the container with id 294f0519043981657168711190701420bceca0c21511219c3a8eadfaa6e6edb0
Sep 17 23:23:57.894996 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:57.894960    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd" event={"ID":"f6e4f647-6569-4236-aabb-ebd6aab162f2","Type":"ContainerStarted","Data":"fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96"}
Sep 17 23:23:57.895440 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:57.895030    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:23:57.895440 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:57.895054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd" event={"ID":"f6e4f647-6569-4236-aabb-ebd6aab162f2","Type":"ContainerStarted","Data":"294f0519043981657168711190701420bceca0c21511219c3a8eadfaa6e6edb0"}
Sep 17 23:23:57.912242 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:57.912188    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd" podStartSLOduration=1.912172069 podStartE2EDuration="1.912172069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:23:57.910963476 +0000 UTC m=+1057.915155316" watchObservedRunningTime="2026-09-17 23:23:57.912172069 +0000 UTC m=+1057.916364167"
Sep 17 23:23:58.608431 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.608385    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64" podUID="235942d0-b7cd-43a8-aa83-780ed533779a" containerName="main" containerID="cri-o://bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e" gracePeriod=45
Sep 17 23:23:58.866944 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.866915    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:58.903922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.903884    2581 generic.go:356] "Generic (PLEG): container finished" podID="235942d0-b7cd-43a8-aa83-780ed533779a" containerID="bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e" exitCode=0
Sep 17 23:23:58.904388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.903932    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"
Sep 17 23:23:58.904388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.903963    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64" event={"ID":"235942d0-b7cd-43a8-aa83-780ed533779a","Type":"ContainerDied","Data":"bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e"}
Sep 17 23:23:58.904388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.904010    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64" event={"ID":"235942d0-b7cd-43a8-aa83-780ed533779a","Type":"ContainerDied","Data":"be708661040a013765957d9da460e4d8fa0ba488d42b9769f2c3a4054f8b28a3"}
Sep 17 23:23:58.904388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.904036    2581 scope.go:117] "RemoveContainer" containerID="bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e"
Sep 17 23:23:58.913315 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.913285    2581 scope.go:117] "RemoveContainer" containerID="bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e"
Sep 17 23:23:58.913597 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:23:58.913578    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e\": container with ID starting with bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e not found: ID does not exist" containerID="bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e"
Sep 17 23:23:58.913652 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.913606    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e"} err="failed to get container status \"bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e\": rpc error: code = NotFound desc = could not find container \"bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e\": container with ID starting with bbd2d90fba4242bbfae8056a63e4068abfa87c6e5d19d4caf646f41b7dd96c1e not found: ID does not exist"
Sep 17 23:23:58.981001 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.980973    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-home\") pod \"235942d0-b7cd-43a8-aa83-780ed533779a\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") "
Sep 17 23:23:58.981189 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981074    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-dshm\") pod \"235942d0-b7cd-43a8-aa83-780ed533779a\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") "
Sep 17 23:23:58.981189 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981107    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-model-cache\") pod \"235942d0-b7cd-43a8-aa83-780ed533779a\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") "
Sep 17 23:23:58.981189 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981178    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/235942d0-b7cd-43a8-aa83-780ed533779a-tls-certs\") pod \"235942d0-b7cd-43a8-aa83-780ed533779a\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") "
Sep 17 23:23:58.981364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981236    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwzwq\" (UniqueName: \"kubernetes.io/projected/235942d0-b7cd-43a8-aa83-780ed533779a-kube-api-access-qwzwq\") pod \"235942d0-b7cd-43a8-aa83-780ed533779a\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") "
Sep 17 23:23:58.981364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981252    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-home" (OuterVolumeSpecName: "home") pod "235942d0-b7cd-43a8-aa83-780ed533779a" (UID: "235942d0-b7cd-43a8-aa83-780ed533779a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:58.981364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981271    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-tmp-dir\") pod \"235942d0-b7cd-43a8-aa83-780ed533779a\" (UID: \"235942d0-b7cd-43a8-aa83-780ed533779a\") "
Sep 17 23:23:58.981507 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981403    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-model-cache" (OuterVolumeSpecName: "model-cache") pod "235942d0-b7cd-43a8-aa83-780ed533779a" (UID: "235942d0-b7cd-43a8-aa83-780ed533779a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:58.981632 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981603    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "235942d0-b7cd-43a8-aa83-780ed533779a" (UID: "235942d0-b7cd-43a8-aa83-780ed533779a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:58.981772 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981751    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:23:58.981836 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981778    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:23:58.981836 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.981791    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:23:58.983352 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.983331    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-dshm" (OuterVolumeSpecName: "dshm") pod "235942d0-b7cd-43a8-aa83-780ed533779a" (UID: "235942d0-b7cd-43a8-aa83-780ed533779a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:23:58.983718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.983700    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/235942d0-b7cd-43a8-aa83-780ed533779a-kube-api-access-qwzwq" (OuterVolumeSpecName: "kube-api-access-qwzwq") pod "235942d0-b7cd-43a8-aa83-780ed533779a" (UID: "235942d0-b7cd-43a8-aa83-780ed533779a"). InnerVolumeSpecName "kube-api-access-qwzwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:23:58.983718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:58.983705    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/235942d0-b7cd-43a8-aa83-780ed533779a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "235942d0-b7cd-43a8-aa83-780ed533779a" (UID: "235942d0-b7cd-43a8-aa83-780ed533779a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:23:59.082763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:59.082734    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/235942d0-b7cd-43a8-aa83-780ed533779a-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:23:59.082763 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:59.082761    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/235942d0-b7cd-43a8-aa83-780ed533779a-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:23:59.082927 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:59.082771    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwzwq\" (UniqueName: \"kubernetes.io/projected/235942d0-b7cd-43a8-aa83-780ed533779a-kube-api-access-qwzwq\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:23:59.226791 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:59.226762    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"]
Sep 17 23:23:59.230335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:59.230312    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-b6f859/noten-model-b6f859-kserve-68f9dcb88c-lct64"]
Sep 17 23:23:59.994382 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:23:59.994346    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="235942d0-b7cd-43a8-aa83-780ed533779a" path="/var/lib/kubelet/pods/235942d0-b7cd-43a8-aa83-780ed533779a/volumes"
Sep 17 23:24:08.906937 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:08.906909    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:24:19.633437 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.633400    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"]
Sep 17 23:24:19.633862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.633849    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="235942d0-b7cd-43a8-aa83-780ed533779a" containerName="main"
Sep 17 23:24:19.633916 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.633864    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="235942d0-b7cd-43a8-aa83-780ed533779a" containerName="main"
Sep 17 23:24:19.633964 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.633954    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="235942d0-b7cd-43a8-aa83-780ed533779a" containerName="main"
Sep 17 23:24:19.636733 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.636717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"
Sep 17 23:24:19.639479 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.639458    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:24:19.639583 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.639501    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:24:19.645894 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.645860    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"]
Sep 17 23:24:19.696843 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.696803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbacee73-637e-4da4-a4fa-fb824f4afa05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7aa654-w454l\" (UID: \"cbacee73-637e-4da4-a4fa-fb824f4afa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"
Sep 17 23:24:19.797978 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.797937    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbacee73-637e-4da4-a4fa-fb824f4afa05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7aa654-w454l\" (UID: \"cbacee73-637e-4da4-a4fa-fb824f4afa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"
Sep 17 23:24:19.798620 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.798601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbacee73-637e-4da4-a4fa-fb824f4afa05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7aa654-w454l\" (UID: \"cbacee73-637e-4da4-a4fa-fb824f4afa05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"
Sep 17 23:24:19.948792 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:19.948707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"
Sep 17 23:24:20.225046 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:20.225015    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"]
Sep 17 23:24:20.225918 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:24:20.225893    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbacee73_637e_4da4_a4fa_fb824f4afa05.slice/crio-4d3c9fd276270cf78ea74c4e9a64c50ed8a3a3bb76c185ecdcb5bb8f8037a0d7 WatchSource:0}: Error finding container 4d3c9fd276270cf78ea74c4e9a64c50ed8a3a3bb76c185ecdcb5bb8f8037a0d7: Status 404 returned error can't find the container with id 4d3c9fd276270cf78ea74c4e9a64c50ed8a3a3bb76c185ecdcb5bb8f8037a0d7
Sep 17 23:24:21.049958 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:21.049924    2581 generic.go:356] "Generic (PLEG): container finished" podID="cbacee73-637e-4da4-a4fa-fb824f4afa05" containerID="97a96317f010eaaed65875e8da0e31eeb1583bf3b8cdbd0c7deaf17d86098e97" exitCode=0
Sep 17 23:24:21.050364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:21.050011    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l" event={"ID":"cbacee73-637e-4da4-a4fa-fb824f4afa05","Type":"ContainerDied","Data":"97a96317f010eaaed65875e8da0e31eeb1583bf3b8cdbd0c7deaf17d86098e97"}
Sep 17 23:24:21.050364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:21.050060    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l" event={"ID":"cbacee73-637e-4da4-a4fa-fb824f4afa05","Type":"ContainerStarted","Data":"4d3c9fd276270cf78ea74c4e9a64c50ed8a3a3bb76c185ecdcb5bb8f8037a0d7"}
Sep 17 23:24:22.198364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:22.198341    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"
Sep 17 23:24:22.221114 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:22.221089    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbacee73-637e-4da4-a4fa-fb824f4afa05-maas-api-service-ca\") pod \"cbacee73-637e-4da4-a4fa-fb824f4afa05\" (UID: \"cbacee73-637e-4da4-a4fa-fb824f4afa05\") "
Sep 17 23:24:22.221410 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:22.221384    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbacee73-637e-4da4-a4fa-fb824f4afa05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cbacee73-637e-4da4-a4fa-fb824f4afa05" (UID: "cbacee73-637e-4da4-a4fa-fb824f4afa05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:24:22.221543 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:22.221524    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbacee73-637e-4da4-a4fa-fb824f4afa05-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:23.063239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:23.063210    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"
Sep 17 23:24:23.063417 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:23.063257    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l" event={"ID":"cbacee73-637e-4da4-a4fa-fb824f4afa05","Type":"ContainerDied","Data":"4d3c9fd276270cf78ea74c4e9a64c50ed8a3a3bb76c185ecdcb5bb8f8037a0d7"}
Sep 17 23:24:23.063417 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:23.063311    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d3c9fd276270cf78ea74c4e9a64c50ed8a3a3bb76c185ecdcb5bb8f8037a0d7"
Sep 17 23:24:30.453352 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.453313    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"]
Sep 17 23:24:30.453974 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.453552    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs" podUID="7d20af4f-15b8-4a91-bc0c-40ba45a64ccf" containerName="payload-pre-processing" containerID="cri-o://1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903" gracePeriod=30
Sep 17 23:24:30.629705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.629642    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"]
Sep 17 23:24:30.632280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.632251    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7aa654-w454l"]
Sep 17 23:24:30.735224 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.730674    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:24:30.812909 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.812873    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hms87\" (UniqueName: \"kubernetes.io/projected/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-kube-api-access-hms87\") pod \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\" (UID: \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\") "
Sep 17 23:24:30.813109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.813022    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-plugins-config-volume\") pod \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\" (UID: \"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf\") "
Sep 17 23:24:30.813383 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.813358    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d20af4f-15b8-4a91-bc0c-40ba45a64ccf" (UID: "7d20af4f-15b8-4a91-bc0c-40ba45a64ccf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:24:30.815051 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.815018    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-kube-api-access-hms87" (OuterVolumeSpecName: "kube-api-access-hms87") pod "7d20af4f-15b8-4a91-bc0c-40ba45a64ccf" (UID: "7d20af4f-15b8-4a91-bc0c-40ba45a64ccf"). InnerVolumeSpecName "kube-api-access-hms87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:24:30.914250 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.914216    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hms87\" (UniqueName: \"kubernetes.io/projected/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-kube-api-access-hms87\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:30.914250 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:30.914246    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:31.125821 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.125782    2581 generic.go:356] "Generic (PLEG): container finished" podID="7d20af4f-15b8-4a91-bc0c-40ba45a64ccf" containerID="1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903" exitCode=0
Sep 17 23:24:31.126030 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.125838    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"
Sep 17 23:24:31.126030 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.125889    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs" event={"ID":"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf","Type":"ContainerDied","Data":"1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903"}
Sep 17 23:24:31.126030 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.125978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs" event={"ID":"7d20af4f-15b8-4a91-bc0c-40ba45a64ccf","Type":"ContainerDied","Data":"15f9539925d9bbfc1549e4e681c82d4a62ce946aa5eb0919ce0bce20f776d03a"}
Sep 17 23:24:31.126030 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.126011    2581 scope.go:117] "RemoveContainer" containerID="1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903"
Sep 17 23:24:31.136259 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.136240    2581 scope.go:117] "RemoveContainer" containerID="1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903"
Sep 17 23:24:31.136526 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:24:31.136508    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903\": container with ID starting with 1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903 not found: ID does not exist" containerID="1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903"
Sep 17 23:24:31.136575 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.136536    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903"} err="failed to get container status \"1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903\": rpc error: code = NotFound desc = could not find container \"1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903\": container with ID starting with 1c3e771ea8b7a019213ee694c17765ea182d2d74990a6ec0879e9c7ee73ee903 not found: ID does not exist"
Sep 17 23:24:31.155549 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.155516    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"]
Sep 17 23:24:31.156814 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.156787    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7aa654-7d4c68d78-jt6cs"]
Sep 17 23:24:31.984572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.984539    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"]
Sep 17 23:24:31.985230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.985209    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbacee73-637e-4da4-a4fa-fb824f4afa05" containerName="revoke-keys"
Sep 17 23:24:31.985230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.985231    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbacee73-637e-4da4-a4fa-fb824f4afa05" containerName="revoke-keys"
Sep 17 23:24:31.985388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.985283    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d20af4f-15b8-4a91-bc0c-40ba45a64ccf" containerName="payload-pre-processing"
Sep 17 23:24:31.985388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.985293    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d20af4f-15b8-4a91-bc0c-40ba45a64ccf" containerName="payload-pre-processing"
Sep 17 23:24:31.985487 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.985388    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbacee73-637e-4da4-a4fa-fb824f4afa05" containerName="revoke-keys"
Sep 17 23:24:31.985487 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.985402    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d20af4f-15b8-4a91-bc0c-40ba45a64ccf" containerName="payload-pre-processing"
Sep 17 23:24:31.990646 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.990626    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:31.993048 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.993028    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6824f488-openshift-default-dockercfg-d6tnp\""
Sep 17 23:24:31.998757 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.998730    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d20af4f-15b8-4a91-bc0c-40ba45a64ccf" path="/var/lib/kubelet/pods/7d20af4f-15b8-4a91-bc0c-40ba45a64ccf/volumes"
Sep 17 23:24:31.999335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:31.999283    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbacee73-637e-4da4-a4fa-fb824f4afa05" path="/var/lib/kubelet/pods/cbacee73-637e-4da4-a4fa-fb824f4afa05/volumes"
Sep 17 23:24:32.003193 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.003171    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"]
Sep 17 23:24:32.127971 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.127930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-certs\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.127971 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.127976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-socket\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.128279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.128012    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-credential-socket\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.128279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.128086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbgpt\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-kube-api-access-jbgpt\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.128279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.128152    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-data\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.128279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.128177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-token\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.128279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.128222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istiod-ca-cert\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.128279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.128255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-podinfo\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.128511 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.128320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-envoy\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229117 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-data\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229117 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-token\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229203    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istiod-ca-cert\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229391 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-podinfo\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229505 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-envoy\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229586 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229558    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-certs\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229586 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-data\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229701 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-socket\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229701 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229645    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-credential-socket\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229701 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbgpt\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-kube-api-access-jbgpt\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229883 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229836    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-certs\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.229962 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-socket\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.230020 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.229984    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istiod-ca-cert\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.230020 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.230007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-credential-socket\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.231621 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.231600    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-envoy\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.232183 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.232166    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-podinfo\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.237464 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.237409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-token\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.237951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.237934    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbgpt\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-kube-api-access-jbgpt\") pod \"e2e-mt-6824f488-openshift-default-846cb44784-qxw7x\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") " pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.303705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.303671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:32.585246 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.585215    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"]
Sep 17 23:24:32.586154 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:24:32.586110    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d2a174_f8d5_4f5e_9e65_a0919f106b69.slice/crio-3d42644ce37b9a5f023351c364579de34111a2ae607c5c1fb4b5117973d7e2d8 WatchSource:0}: Error finding container 3d42644ce37b9a5f023351c364579de34111a2ae607c5c1fb4b5117973d7e2d8: Status 404 returned error can't find the container with id 3d42644ce37b9a5f023351c364579de34111a2ae607c5c1fb4b5117973d7e2d8
Sep 17 23:24:32.588744 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.588708    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:24:32.588813 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.588801    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:24:32.588861 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:32.588844    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:24:33.140332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:33.140296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" event={"ID":"15d2a174-f8d5-4f5e-9e65-a0919f106b69","Type":"ContainerStarted","Data":"9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588"}
Sep 17 23:24:33.140720 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:33.140350    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" event={"ID":"15d2a174-f8d5-4f5e-9e65-a0919f106b69","Type":"ContainerStarted","Data":"3d42644ce37b9a5f023351c364579de34111a2ae607c5c1fb4b5117973d7e2d8"}
Sep 17 23:24:33.164280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:33.164220    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podStartSLOduration=2.16420039 podStartE2EDuration="2.16420039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:24:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:24:33.159202743 +0000 UTC m=+1093.163394584" watchObservedRunningTime="2026-09-17 23:24:33.16420039 +0000 UTC m=+1093.168392231"
Sep 17 23:24:33.304218 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:33.304180    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:33.305558 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:33.305531    2581 patch_prober.go:28] interesting pod/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 17 23:24:33.305688 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:33.305581    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 17 23:24:34.308040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:34.308010    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:35.153019 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:35.152988    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:35.154146 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:35.154094    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:24:37.361320 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.361283    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"]
Sep 17 23:24:37.361746 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.361534    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy" containerID="cri-o://c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225" gracePeriod=30
Sep 17 23:24:37.688338 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.688258    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"]
Sep 17 23:24:37.692074 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.692053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"
Sep 17 23:24:37.694697 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.694677    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:24:37.694697 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.694689    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:24:37.700604 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.700367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"]
Sep 17 23:24:37.792287 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.792255    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8392d5a3-e429-454a-ba5e-064e04e45b43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8\" (UID: \"8392d5a3-e429-454a-ba5e-064e04e45b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"
Sep 17 23:24:37.893659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.893624    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8392d5a3-e429-454a-ba5e-064e04e45b43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8\" (UID: \"8392d5a3-e429-454a-ba5e-064e04e45b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"
Sep 17 23:24:37.894308 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:37.894290    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8392d5a3-e429-454a-ba5e-064e04e45b43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8\" (UID: \"8392d5a3-e429-454a-ba5e-064e04e45b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"
Sep 17 23:24:38.004590 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:38.004513    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"
Sep 17 23:24:38.225144 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:38.224913    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"]
Sep 17 23:24:38.278404 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:38.278374    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"]
Sep 17 23:24:38.279615 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:24:38.279591    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8392d5a3_e429_454a_ba5e_064e04e45b43.slice/crio-0c38649ebce58eea26d4ed3944c5b6c22d3864d0436fb6e62a566a464dfd1f8f WatchSource:0}: Error finding container 0c38649ebce58eea26d4ed3944c5b6c22d3864d0436fb6e62a566a464dfd1f8f: Status 404 returned error can't find the container with id 0c38649ebce58eea26d4ed3944c5b6c22d3864d0436fb6e62a566a464dfd1f8f
Sep 17 23:24:39.180859 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:39.180822    2581 generic.go:356] "Generic (PLEG): container finished" podID="8392d5a3-e429-454a-ba5e-064e04e45b43" containerID="3f9dfbc5914c26e76c76b598edd1f857eaa9beba48f3195dfe4e80767b90961a" exitCode=0
Sep 17 23:24:39.181311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:39.180911    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8" event={"ID":"8392d5a3-e429-454a-ba5e-064e04e45b43","Type":"ContainerDied","Data":"3f9dfbc5914c26e76c76b598edd1f857eaa9beba48f3195dfe4e80767b90961a"}
Sep 17 23:24:39.181311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:39.180976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8" event={"ID":"8392d5a3-e429-454a-ba5e-064e04e45b43","Type":"ContainerStarted","Data":"0c38649ebce58eea26d4ed3944c5b6c22d3864d0436fb6e62a566a464dfd1f8f"}
Sep 17 23:24:40.329962 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.329931    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"
Sep 17 23:24:40.418892 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.418861    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8392d5a3-e429-454a-ba5e-064e04e45b43-maas-api-service-ca\") pod \"8392d5a3-e429-454a-ba5e-064e04e45b43\" (UID: \"8392d5a3-e429-454a-ba5e-064e04e45b43\") "
Sep 17 23:24:40.419233 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.419207    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8392d5a3-e429-454a-ba5e-064e04e45b43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8392d5a3-e429-454a-ba5e-064e04e45b43" (UID: "8392d5a3-e429-454a-ba5e-064e04e45b43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:24:40.520356 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.520278    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8392d5a3-e429-454a-ba5e-064e04e45b43-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:40.836633 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.836548    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"]
Sep 17 23:24:40.837066 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.837048    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8392d5a3-e429-454a-ba5e-064e04e45b43" containerName="revoke-keys"
Sep 17 23:24:40.837192 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.837069    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8392d5a3-e429-454a-ba5e-064e04e45b43" containerName="revoke-keys"
Sep 17 23:24:40.837192 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.837180    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8392d5a3-e429-454a-ba5e-064e04e45b43" containerName="revoke-keys"
Sep 17 23:24:40.840385 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.840367    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:40.843016 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.842998    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6824f488-dockercfg-7rwsj\""
Sep 17 23:24:40.843266 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.843244    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6824f488\""
Sep 17 23:24:40.851554 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.851530    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"]
Sep 17 23:24:40.925997 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.925956    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2nq2\" (UniqueName: \"kubernetes.io/projected/c166e699-5ca2-4747-8590-e9a7511f2a4f-kube-api-access-f2nq2\") pod \"payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2\" (UID: \"c166e699-5ca2-4747-8590-e9a7511f2a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:40.926454 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:40.926420    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c166e699-5ca2-4747-8590-e9a7511f2a4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2\" (UID: \"c166e699-5ca2-4747-8590-e9a7511f2a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:41.028018 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.027973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2nq2\" (UniqueName: \"kubernetes.io/projected/c166e699-5ca2-4747-8590-e9a7511f2a4f-kube-api-access-f2nq2\") pod \"payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2\" (UID: \"c166e699-5ca2-4747-8590-e9a7511f2a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:41.028220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.028087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c166e699-5ca2-4747-8590-e9a7511f2a4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2\" (UID: \"c166e699-5ca2-4747-8590-e9a7511f2a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:41.028781 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.028760    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c166e699-5ca2-4747-8590-e9a7511f2a4f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2\" (UID: \"c166e699-5ca2-4747-8590-e9a7511f2a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:41.037252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.037227    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2nq2\" (UniqueName: \"kubernetes.io/projected/c166e699-5ca2-4747-8590-e9a7511f2a4f-kube-api-access-f2nq2\") pod \"payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2\" (UID: \"c166e699-5ca2-4747-8590-e9a7511f2a4f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:41.152037 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.152003    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:41.201946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.201920    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"
Sep 17 23:24:41.202092 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.201948    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8" event={"ID":"8392d5a3-e429-454a-ba5e-064e04e45b43","Type":"ContainerDied","Data":"0c38649ebce58eea26d4ed3944c5b6c22d3864d0436fb6e62a566a464dfd1f8f"}
Sep 17 23:24:41.202092 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.202008    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c38649ebce58eea26d4ed3944c5b6c22d3864d0436fb6e62a566a464dfd1f8f"
Sep 17 23:24:41.436199 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:41.436169    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"]
Sep 17 23:24:41.436522 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:24:41.436407    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc166e699_5ca2_4747_8590_e9a7511f2a4f.slice/crio-875c34336b898e7fdece52008d18ddc4b382fe7c8567d0e9630ccc437dbc7a47 WatchSource:0}: Error finding container 875c34336b898e7fdece52008d18ddc4b382fe7c8567d0e9630ccc437dbc7a47: Status 404 returned error can't find the container with id 875c34336b898e7fdece52008d18ddc4b382fe7c8567d0e9630ccc437dbc7a47
Sep 17 23:24:42.216272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.215390    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2" event={"ID":"c166e699-5ca2-4747-8590-e9a7511f2a4f","Type":"ContainerStarted","Data":"e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4"}
Sep 17 23:24:42.216272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.215456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2" event={"ID":"c166e699-5ca2-4747-8590-e9a7511f2a4f","Type":"ContainerStarted","Data":"875c34336b898e7fdece52008d18ddc4b382fe7c8567d0e9630ccc437dbc7a47"}
Sep 17 23:24:42.216272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.216229    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:42.235983 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.235933    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2" podStartSLOduration=2.235917404 podStartE2EDuration="2.235917404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:24:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:24:42.232636374 +0000 UTC m=+1102.236828215" watchObservedRunningTime="2026-09-17 23:24:42.235917404 +0000 UTC m=+1102.240109242"
Sep 17 23:24:42.629535 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.629511    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:24:42.746671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746584    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-socket\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746622    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-data\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746651    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-credential-socket\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746729    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-envoy\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746744    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-certs\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746798    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2700b74-04f8-475e-9000-057919d586a3-istiod-ca-cert\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746837    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhtfq\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-kube-api-access-dhtfq\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746900    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-istio-token\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746925    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2700b74-04f8-475e-9000-057919d586a3-istio-podinfo\") pod \"f2700b74-04f8-475e-9000-057919d586a3\" (UID: \"f2700b74-04f8-475e-9000-057919d586a3\") "
Sep 17 23:24:42.746956 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746935    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:42.747365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.746949    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-data" (OuterVolumeSpecName: "istio-data") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:42.747365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.747026    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:42.747365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.747226    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:42.747365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.747340    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2700b74-04f8-475e-9000-057919d586a3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:24:42.747538 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.747420    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:42.747538 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.747438    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-data\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:42.747538 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.747452    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-credential-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:42.747538 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.747466    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-workload-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:42.749239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.749217    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:42.749604 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.749580    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-kube-api-access-dhtfq" (OuterVolumeSpecName: "kube-api-access-dhtfq") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "kube-api-access-dhtfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:24:42.749716 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.749664    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:24:42.749775 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.749731    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2700b74-04f8-475e-9000-057919d586a3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f2700b74-04f8-475e-9000-057919d586a3" (UID: "f2700b74-04f8-475e-9000-057919d586a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:24:42.848721 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.848686    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2700b74-04f8-475e-9000-057919d586a3-istio-envoy\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:42.848721 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.848715    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2700b74-04f8-475e-9000-057919d586a3-istiod-ca-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:42.848721 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.848727    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhtfq\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-kube-api-access-dhtfq\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:42.848942 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.848737    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2700b74-04f8-475e-9000-057919d586a3-istio-token\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:42.848942 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:42.848746    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2700b74-04f8-475e-9000-057919d586a3-istio-podinfo\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:43.223158 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.223103    2581 generic.go:356] "Generic (PLEG): container finished" podID="f2700b74-04f8-475e-9000-057919d586a3" containerID="c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225" exitCode=0
Sep 17 23:24:43.223366 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.223156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" event={"ID":"f2700b74-04f8-475e-9000-057919d586a3","Type":"ContainerDied","Data":"c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225"}
Sep 17 23:24:43.223366 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.223179    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"
Sep 17 23:24:43.223366 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.223231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx" event={"ID":"f2700b74-04f8-475e-9000-057919d586a3","Type":"ContainerDied","Data":"6c722b11acf6d7329cd1161b52d135295956a9475cbfac275001d37c05b2002b"}
Sep 17 23:24:43.223366 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.223264    2581 scope.go:117] "RemoveContainer" containerID="c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225"
Sep 17 23:24:43.234141 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.234111    2581 scope.go:117] "RemoveContainer" containerID="04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a"
Sep 17 23:24:43.243792 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.243772    2581 scope.go:117] "RemoveContainer" containerID="c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225"
Sep 17 23:24:43.244090 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:24:43.244073    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225\": container with ID starting with c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225 not found: ID does not exist" containerID="c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225"
Sep 17 23:24:43.244188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.244098    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225"} err="failed to get container status \"c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225\": rpc error: code = NotFound desc = could not find container \"c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225\": container with ID starting with c546a217587e3ac852237acbf0c36fe0959b45f7bfdae9a6a6bee83722191225 not found: ID does not exist"
Sep 17 23:24:43.244188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.244116    2581 scope.go:117] "RemoveContainer" containerID="04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a"
Sep 17 23:24:43.244470 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:24:43.244440    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a\": container with ID starting with 04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a not found: ID does not exist" containerID="04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a"
Sep 17 23:24:43.244521 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.244469    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a"} err="failed to get container status \"04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a\": rpc error: code = NotFound desc = could not find container \"04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a\": container with ID starting with 04e6bde430554bb59f4f5437f6fa8111a5ace8388a2eeee6d5ddedf914aed29a not found: ID does not exist"
Sep 17 23:24:43.246552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.246534    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"]
Sep 17 23:24:43.250915 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.250894    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7aa654-openshift-default-5bb856cfd6-hqtbx"]
Sep 17 23:24:43.994179 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:43.994142    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2700b74-04f8-475e-9000-057919d586a3" path="/var/lib/kubelet/pods/f2700b74-04f8-475e-9000-057919d586a3/volumes"
Sep 17 23:24:48.599662 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:48.599625    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"]
Sep 17 23:24:48.602490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:48.602467    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8fc0d3-njhd8"]
Sep 17 23:24:49.993899 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:49.993858    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8392d5a3-e429-454a-ba5e-064e04e45b43" path="/var/lib/kubelet/pods/8392d5a3-e429-454a-ba5e-064e04e45b43/volumes"
Sep 17 23:24:53.245306 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.245262    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr" podUID="f3f7566d-3262-4401-8b85-96e49fdd6b05" containerName="main" containerID="cri-o://f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52" gracePeriod=45
Sep 17 23:24:53.503486 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.503416    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:24:53.558984 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.558953    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-dshm\") pod \"f3f7566d-3262-4401-8b85-96e49fdd6b05\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") "
Sep 17 23:24:53.559143 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559002    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24zpn\" (UniqueName: \"kubernetes.io/projected/f3f7566d-3262-4401-8b85-96e49fdd6b05-kube-api-access-24zpn\") pod \"f3f7566d-3262-4401-8b85-96e49fdd6b05\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") "
Sep 17 23:24:53.559143 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559057    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-model-cache\") pod \"f3f7566d-3262-4401-8b85-96e49fdd6b05\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") "
Sep 17 23:24:53.559246 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559143    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-tmp-dir\") pod \"f3f7566d-3262-4401-8b85-96e49fdd6b05\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") "
Sep 17 23:24:53.559246 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559224    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-home\") pod \"f3f7566d-3262-4401-8b85-96e49fdd6b05\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") "
Sep 17 23:24:53.559351 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559273    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f7566d-3262-4401-8b85-96e49fdd6b05-tls-certs\") pod \"f3f7566d-3262-4401-8b85-96e49fdd6b05\" (UID: \"f3f7566d-3262-4401-8b85-96e49fdd6b05\") "
Sep 17 23:24:53.559408 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559362    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3f7566d-3262-4401-8b85-96e49fdd6b05" (UID: "f3f7566d-3262-4401-8b85-96e49fdd6b05"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:53.559490 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559466    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3f7566d-3262-4401-8b85-96e49fdd6b05" (UID: "f3f7566d-3262-4401-8b85-96e49fdd6b05"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:53.559552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559532    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-home" (OuterVolumeSpecName: "home") pod "f3f7566d-3262-4401-8b85-96e49fdd6b05" (UID: "f3f7566d-3262-4401-8b85-96e49fdd6b05"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:53.559666 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559649    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:53.559728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559672    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:53.559728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.559687    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:53.561395 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.561375    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-dshm" (OuterVolumeSpecName: "dshm") pod "f3f7566d-3262-4401-8b85-96e49fdd6b05" (UID: "f3f7566d-3262-4401-8b85-96e49fdd6b05"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:24:53.561687 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.561662    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3f7566d-3262-4401-8b85-96e49fdd6b05-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3f7566d-3262-4401-8b85-96e49fdd6b05" (UID: "f3f7566d-3262-4401-8b85-96e49fdd6b05"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:24:53.561782 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.561688    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3f7566d-3262-4401-8b85-96e49fdd6b05-kube-api-access-24zpn" (OuterVolumeSpecName: "kube-api-access-24zpn") pod "f3f7566d-3262-4401-8b85-96e49fdd6b05" (UID: "f3f7566d-3262-4401-8b85-96e49fdd6b05"). InnerVolumeSpecName "kube-api-access-24zpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:24:53.660284 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.660246    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3f7566d-3262-4401-8b85-96e49fdd6b05-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:53.660284 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.660274    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3f7566d-3262-4401-8b85-96e49fdd6b05-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:53.660284 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:53.660284    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24zpn\" (UniqueName: \"kubernetes.io/projected/f3f7566d-3262-4401-8b85-96e49fdd6b05-kube-api-access-24zpn\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:24:54.232524 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.232495    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:24:54.300957 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.300863    2581 generic.go:356] "Generic (PLEG): container finished" podID="f3f7566d-3262-4401-8b85-96e49fdd6b05" containerID="f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52" exitCode=0
Sep 17 23:24:54.300957 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.300916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr" event={"ID":"f3f7566d-3262-4401-8b85-96e49fdd6b05","Type":"ContainerDied","Data":"f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52"}
Sep 17 23:24:54.300957 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.300928    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"
Sep 17 23:24:54.300957 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.300957    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr" event={"ID":"f3f7566d-3262-4401-8b85-96e49fdd6b05","Type":"ContainerDied","Data":"f7376582d0c82d10241bff9af8d1c3f93bc6921b7ccb4af12c77fe0b19f71f65"}
Sep 17 23:24:54.301578 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.300979    2581 scope.go:117] "RemoveContainer" containerID="f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52"
Sep 17 23:24:54.310961 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.310941    2581 scope.go:117] "RemoveContainer" containerID="f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52"
Sep 17 23:24:54.311342 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:24:54.311319    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52\": container with ID starting with f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52 not found: ID does not exist" containerID="f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52"
Sep 17 23:24:54.311423 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.311349    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52"} err="failed to get container status \"f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52\": rpc error: code = NotFound desc = could not find container \"f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52\": container with ID starting with f39888afbb081ea0ddf4a5b8a0d6bdffa10f05f2d67d02d8000550497d586b52 not found: ID does not exist"
Sep 17 23:24:54.320222 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.320199    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"]
Sep 17 23:24:54.325566 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:54.325543    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8fc0d3/test-model-8fc0d3-kserve-84dbd845c-w29mr"]
Sep 17 23:24:55.994180 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:55.994150    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3f7566d-3262-4401-8b85-96e49fdd6b05" path="/var/lib/kubelet/pods/f3f7566d-3262-4401-8b85-96e49fdd6b05/volumes"
Sep 17 23:24:56.364967 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:56.364939    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"]
Sep 17 23:24:56.365196 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:56.365166    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy" containerID="cri-o://388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106" gracePeriod=30
Sep 17 23:24:57.331240 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.331206    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"]
Sep 17 23:24:57.331904 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.331877    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3f7566d-3262-4401-8b85-96e49fdd6b05" containerName="main"
Sep 17 23:24:57.331904 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.331905    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3f7566d-3262-4401-8b85-96e49fdd6b05" containerName="main"
Sep 17 23:24:57.332120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.331929    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy"
Sep 17 23:24:57.332120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.331939    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy"
Sep 17 23:24:57.332120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.331950    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy"
Sep 17 23:24:57.332120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.331959    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy"
Sep 17 23:24:57.332340 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.332151    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy"
Sep 17 23:24:57.332340 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.332172    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3f7566d-3262-4401-8b85-96e49fdd6b05" containerName="main"
Sep 17 23:24:57.337312 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.337291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.340301 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.340275    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-e5aae7-openshift-default-dockercfg-p84cr\""
Sep 17 23:24:57.347759 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.347732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"]
Sep 17 23:24:57.397179 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397150    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-credential-socket\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.397179 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397186    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-istio-token\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.397360 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-envoy\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.397360 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-socket\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.397360 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d0b192-a431-443d-a2d2-ca40277114df-istiod-ca-cert\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.397475 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397376    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d0b192-a431-443d-a2d2-ca40277114df-istio-podinfo\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.397475 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-data\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.397542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397479    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-certs\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.397542 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.397497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dctn5\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-kube-api-access-dctn5\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498138 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-credential-socket\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498184    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-istio-token\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498228    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-envoy\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-socket\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498286    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d0b192-a431-443d-a2d2-ca40277114df-istiod-ca-cert\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d0b192-a431-443d-a2d2-ca40277114df-istio-podinfo\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498397    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-data\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-certs\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498564 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dctn5\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-kube-api-access-dctn5\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.498768 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.498654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-socket\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.499057 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.499030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-data\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.499268 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.499238    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-certs\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.499492 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.499471    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d0b192-a431-443d-a2d2-ca40277114df-istiod-ca-cert\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.499557 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.499541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-credential-socket\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.500644 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.500623    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-envoy\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.501067 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.501048    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d0b192-a431-443d-a2d2-ca40277114df-istio-podinfo\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.506794 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.506773    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-istio-token\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.506874 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.506853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dctn5\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-kube-api-access-dctn5\") pod \"e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") " pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.652358 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.652320    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:57.948154 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.948112    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"]
Sep 17 23:24:57.949921 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:24:57.949893    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71d0b192_a431_443d_a2d2_ca40277114df.slice/crio-add44065967cc32bfe8ecc09bccec9f417078f8c8d8d67344e79e7a26062600e WatchSource:0}: Error finding container add44065967cc32bfe8ecc09bccec9f417078f8c8d8d67344e79e7a26062600e: Status 404 returned error can't find the container with id add44065967cc32bfe8ecc09bccec9f417078f8c8d8d67344e79e7a26062600e
Sep 17 23:24:57.952300 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.952270    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:24:57.952370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.952332    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:24:57.952501 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:57.952380    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:24:58.335247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:58.335152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" event={"ID":"71d0b192-a431-443d-a2d2-ca40277114df","Type":"ContainerStarted","Data":"87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c"}
Sep 17 23:24:58.335247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:58.335237    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" event={"ID":"71d0b192-a431-443d-a2d2-ca40277114df","Type":"ContainerStarted","Data":"add44065967cc32bfe8ecc09bccec9f417078f8c8d8d67344e79e7a26062600e"}
Sep 17 23:24:58.358769 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:58.358718    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" podStartSLOduration=1.358702649 podStartE2EDuration="1.358702649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:24:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:24:58.355682035 +0000 UTC m=+1118.359873873" watchObservedRunningTime="2026-09-17 23:24:58.358702649 +0000 UTC m=+1118.362894487"
Sep 17 23:24:58.652862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:58.652834    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:24:59.657681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:24:59.657650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:25:00.349379 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:00.349212    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:25:00.353635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:00.353606    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:25:01.631673 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.631610    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:25:01.744816 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.744784    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-credential-socket\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.744816 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.744815    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-socket\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.745052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.744868    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txpw4\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-kube-api-access-txpw4\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.745052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.744904    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-token\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.745052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.744939    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-certs\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.745247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745113    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:01.745247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745153    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:01.745247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745190    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-podinfo\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.745247 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745219    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e49fee3-bc15-48c1-a33f-062892f5d304-istiod-ca-cert\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.745466 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745280    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-envoy\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.745466 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-data\") pod \"5e49fee3-bc15-48c1-a33f-062892f5d304\" (UID: \"5e49fee3-bc15-48c1-a33f-062892f5d304\") "
Sep 17 23:25:01.745466 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745366    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:01.745676 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745592    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:01.745786 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745762    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-data\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:01.745934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745790    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-credential-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:01.745934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745809    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:01.745934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745824    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-workload-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:01.745934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.745821    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e49fee3-bc15-48c1-a33f-062892f5d304-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:25:01.747369 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.747347    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:25:01.747483 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.747401    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:25:01.747706 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.747688    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:25:01.747928 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.747900    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-kube-api-access-txpw4" (OuterVolumeSpecName: "kube-api-access-txpw4") pod "5e49fee3-bc15-48c1-a33f-062892f5d304" (UID: "5e49fee3-bc15-48c1-a33f-062892f5d304"). InnerVolumeSpecName "kube-api-access-txpw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:25:01.847384 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.847350    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-podinfo\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:01.847384 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.847379    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e49fee3-bc15-48c1-a33f-062892f5d304-istiod-ca-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:01.847384 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.847388    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-envoy\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:01.847610 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.847398    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txpw4\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-kube-api-access-txpw4\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:01.847610 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:01.847408    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e49fee3-bc15-48c1-a33f-062892f5d304-istio-token\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:25:02.373793 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.373756    2581 generic.go:356] "Generic (PLEG): container finished" podID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerID="388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106" exitCode=0
Sep 17 23:25:02.374410 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.374378    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"
Sep 17 23:25:02.375053 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.375029    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" event={"ID":"5e49fee3-bc15-48c1-a33f-062892f5d304","Type":"ContainerDied","Data":"388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106"}
Sep 17 23:25:02.375269 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.375249    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j" event={"ID":"5e49fee3-bc15-48c1-a33f-062892f5d304","Type":"ContainerDied","Data":"a4007b7f0f892f7b979160d5ccb2179e453bea7c8c08618224e50423303b10b7"}
Sep 17 23:25:02.375383 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.375295    2581 scope.go:117] "RemoveContainer" containerID="388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106"
Sep 17 23:25:02.388208 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.388188    2581 scope.go:117] "RemoveContainer" containerID="b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726"
Sep 17 23:25:02.396715 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.396687    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"]
Sep 17 23:25:02.397433 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.397410    2581 scope.go:117] "RemoveContainer" containerID="388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106"
Sep 17 23:25:02.398001 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:25:02.397910    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106\": container with ID starting with 388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106 not found: ID does not exist" containerID="388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106"
Sep 17 23:25:02.398001 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.397946    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106"} err="failed to get container status \"388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106\": rpc error: code = NotFound desc = could not find container \"388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106\": container with ID starting with 388b6c3b0eaf45e85f9f1b39000d8c697e75f1c430fa55661e48c10c929b0106 not found: ID does not exist"
Sep 17 23:25:02.398001 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.397968    2581 scope.go:117] "RemoveContainer" containerID="b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726"
Sep 17 23:25:02.398547 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:25:02.398519    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726\": container with ID starting with b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726 not found: ID does not exist" containerID="b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726"
Sep 17 23:25:02.398622 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.398558    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726"} err="failed to get container status \"b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726\": rpc error: code = NotFound desc = could not find container \"b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726\": container with ID starting with b13a0374f5b66b32a73182cc6f25db0d53e73739b90c21cb14d6f24932e6d726 not found: ID does not exist"
Sep 17 23:25:02.400650 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:02.400626    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-8fc0d3-openshift-default-db59cbf8d-l4z2j"]
Sep 17 23:25:03.993915 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:03.993865    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" path="/var/lib/kubelet/pods/5e49fee3-bc15-48c1-a33f-062892f5d304/volumes"
Sep 17 23:25:05.268205 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.266706    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"]
Sep 17 23:25:05.274302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.272610    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy"
Sep 17 23:25:05.274302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.272642    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy"
Sep 17 23:25:05.274302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.272666    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy"
Sep 17 23:25:05.274302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.272675    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy"
Sep 17 23:25:05.274302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.272797    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy"
Sep 17 23:25:05.274302 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.272813    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2700b74-04f8-475e-9000-057919d586a3" containerName="istio-proxy"
Sep 17 23:25:05.279383 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.278603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:05.281897 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.281859    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"]
Sep 17 23:25:05.282230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.282209    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-e5aae7\""
Sep 17 23:25:05.394069 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.394035    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df3b28ed-0290-43bd-a3d2-d4a56bec224d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv\" (UID: \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:05.394586 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.394559    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bckjg\" (UniqueName: \"kubernetes.io/projected/df3b28ed-0290-43bd-a3d2-d4a56bec224d-kube-api-access-bckjg\") pod \"payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv\" (UID: \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:05.495609 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.495572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bckjg\" (UniqueName: \"kubernetes.io/projected/df3b28ed-0290-43bd-a3d2-d4a56bec224d-kube-api-access-bckjg\") pod \"payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv\" (UID: \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:05.495808 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.495635    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df3b28ed-0290-43bd-a3d2-d4a56bec224d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv\" (UID: \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:05.496556 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.496532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df3b28ed-0290-43bd-a3d2-d4a56bec224d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv\" (UID: \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:05.514545 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.514523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bckjg\" (UniqueName: \"kubernetes.io/projected/df3b28ed-0290-43bd-a3d2-d4a56bec224d-kube-api-access-bckjg\") pod \"payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv\" (UID: \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:05.592249 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.592152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:05.869150 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:05.867990    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"]
Sep 17 23:25:05.882896 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:25:05.882000    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3b28ed_0290_43bd_a3d2_d4a56bec224d.slice/crio-d253a2bac1f3df8b84c04252a36b9016234163c62584cdbc21a88b52431f4878 WatchSource:0}: Error finding container d253a2bac1f3df8b84c04252a36b9016234163c62584cdbc21a88b52431f4878: Status 404 returned error can't find the container with id d253a2bac1f3df8b84c04252a36b9016234163c62584cdbc21a88b52431f4878
Sep 17 23:25:06.406543 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:06.406495    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv" event={"ID":"df3b28ed-0290-43bd-a3d2-d4a56bec224d","Type":"ContainerStarted","Data":"93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7"}
Sep 17 23:25:06.407030 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:06.406563    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv" event={"ID":"df3b28ed-0290-43bd-a3d2-d4a56bec224d","Type":"ContainerStarted","Data":"d253a2bac1f3df8b84c04252a36b9016234163c62584cdbc21a88b52431f4878"}
Sep 17 23:25:06.407030 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:06.406610    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:06.424337 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:06.424280    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv" podStartSLOduration=1.4242648039999999 podStartE2EDuration="1.424264804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:25:06.422908671 +0000 UTC m=+1126.427100524" watchObservedRunningTime="2026-09-17 23:25:06.424264804 +0000 UTC m=+1126.428456643"
Sep 17 23:25:10.435952 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:10.435918    2581 generic.go:356] "Generic (PLEG): container finished" podID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerID="9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588" exitCode=0
Sep 17 23:25:10.436361 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:10.435992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" event={"ID":"15d2a174-f8d5-4f5e-9e65-a0919f106b69","Type":"ContainerDied","Data":"9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588"}
Sep 17 23:25:10.436466 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:10.436449    2581 scope.go:117] "RemoveContainer" containerID="9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588"
Sep 17 23:25:10.437778 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:10.437752    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:25:10.437838 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:10.437817    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:25:10.437876 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:10.437847    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:25:11.444171 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:11.444114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" event={"ID":"15d2a174-f8d5-4f5e-9e65-a0919f106b69","Type":"ContainerStarted","Data":"430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f"}
Sep 17 23:25:12.303913 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:12.303875    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:25:12.304155 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:12.304087    2581 patch_prober.go:28] interesting pod/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 17 23:25:12.304234 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:12.304165    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 17 23:25:13.304952 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:13.304910    2581 patch_prober.go:28] interesting pod/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 17 23:25:13.305361 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:13.304972    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 17 23:25:14.304870 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:14.304835    2581 patch_prober.go:28] interesting pod/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 17 23:25:14.305079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:14.304896    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 17 23:25:15.305101 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:15.305041    2581 patch_prober.go:28] interesting pod/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 17 23:25:15.305548 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:15.305155    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 17 23:25:16.304096 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:16.304054    2581 patch_prober.go:28] interesting pod/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Sep 17 23:25:16.304309 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:16.304144    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Sep 17 23:25:17.326961 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:17.326930    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:25:17.327465 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:17.327370    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:25:17.328183 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:17.328160    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:25:17.416751 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:17.416720    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:25:22.181929 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:22.181886    2581 scope.go:117] "RemoveContainer" containerID="e19efae90daafeb8589da24951e6ee05bbadf919269f42576386c7f1e6aa6745"
Sep 17 23:25:22.193573 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:22.193553    2581 scope.go:117] "RemoveContainer" containerID="a92452be304a8b8693870ba66543b39edc92f7ab2e1f1dd75b8ec03f2f4027d9"
Sep 17 23:25:23.542702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:23.542603    2581 generic.go:356] "Generic (PLEG): container finished" podID="71d0b192-a431-443d-a2d2-ca40277114df" containerID="87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c" exitCode=0
Sep 17 23:25:23.542702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:23.542681    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" event={"ID":"71d0b192-a431-443d-a2d2-ca40277114df","Type":"ContainerDied","Data":"87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c"}
Sep 17 23:25:23.543119 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:23.543095    2581 scope.go:117] "RemoveContainer" containerID="87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c"
Sep 17 23:25:23.544332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:23.544302    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:25:23.544423 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:23.544380    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:25:23.544423 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:23.544416    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 17 23:25:24.551271 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:24.551229    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" event={"ID":"71d0b192-a431-443d-a2d2-ca40277114df","Type":"ContainerStarted","Data":"b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b"}
Sep 17 23:25:24.652854 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:24.652822    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:25:24.653033 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:24.653010    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 17 23:25:24.653079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:24.653060    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 17 23:25:25.653293 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:25.653257    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 17 23:25:25.654377 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:25.653865    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 17 23:25:26.653798 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:26.653760    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 17 23:25:26.654421 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:26.654388    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 17 23:25:27.653612 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:27.653572    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 17 23:25:27.653943 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:27.653905    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 17 23:25:28.654457 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:28.654421    2581 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 17 23:25:28.655036 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:28.655004    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 17 23:25:29.797690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:29.797655    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:25:29.798237 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:29.798035    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:25:29.799436 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:29.799413    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:25:30.343907 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.343873    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"]
Sep 17 23:25:30.344435 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.344419    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e49fee3-bc15-48c1-a33f-062892f5d304" containerName="istio-proxy"
Sep 17 23:25:30.347617 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.347601    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.358313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.358286    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-0b4d93\"/\"default-dockercfg-fzhhg\""
Sep 17 23:25:30.358460 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.358332    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-0b4d93\"/\"distinct-2-w0-0b4d93-kserve-self-signed-certs\""
Sep 17 23:25:30.358460 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.358292    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-0b4d93\"/\"kube-root-ca.crt\""
Sep 17 23:25:30.358656 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.358631    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-0b4d93\"/\"openshift-service-ca.crt\""
Sep 17 23:25:30.362039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.362015    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"]
Sep 17 23:25:30.472932 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.472897    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-model-cache\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.473099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.472939    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9mw\" (UniqueName: \"kubernetes.io/projected/09493d74-f4ba-4817-b17c-36a7134a5dfc-kube-api-access-2l9mw\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.473099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.472964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-home\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.473099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.472985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-dshm\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.473242 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.473152    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-tmp-dir\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.473242 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.473231    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09493d74-f4ba-4817-b17c-36a7134a5dfc-tls-certs\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-home\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574335 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-dshm\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574550 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574406    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-tmp-dir\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574550 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09493d74-f4ba-4817-b17c-36a7134a5dfc-tls-certs\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574550 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-model-cache\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574550 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9mw\" (UniqueName: \"kubernetes.io/projected/09493d74-f4ba-4817-b17c-36a7134a5dfc-kube-api-access-2l9mw\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574761 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574735    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-home\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574808 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574775    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-tmp-dir\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.574862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.574848    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-model-cache\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.576736 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.576703    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-dshm\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.576934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.576916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09493d74-f4ba-4817-b17c-36a7134a5dfc-tls-certs\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.593863 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.593834    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9mw\" (UniqueName: \"kubernetes.io/projected/09493d74-f4ba-4817-b17c-36a7134a5dfc-kube-api-access-2l9mw\") pod \"distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") " pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.657913 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.657877    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:30.938617 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:30.938591    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"]
Sep 17 23:25:30.940714 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:25:30.940686    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09493d74_f4ba_4817_b17c_36a7134a5dfc.slice/crio-3dd0b94db264376a3cd2f8be3f4ab34e92e3290565a26e1beb19e380d87ecc26 WatchSource:0}: Error finding container 3dd0b94db264376a3cd2f8be3f4ab34e92e3290565a26e1beb19e380d87ecc26: Status 404 returned error can't find the container with id 3dd0b94db264376a3cd2f8be3f4ab34e92e3290565a26e1beb19e380d87ecc26
Sep 17 23:25:31.608311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:31.608278    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt" event={"ID":"09493d74-f4ba-4817-b17c-36a7134a5dfc","Type":"ContainerStarted","Data":"e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d"}
Sep 17 23:25:31.608509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:31.608321    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt" event={"ID":"09493d74-f4ba-4817-b17c-36a7134a5dfc","Type":"ContainerStarted","Data":"3dd0b94db264376a3cd2f8be3f4ab34e92e3290565a26e1beb19e380d87ecc26"}
Sep 17 23:25:31.626599 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:31.626545    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt" podStartSLOduration=1.6265291400000002 podStartE2EDuration="1.62652914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:25:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:25:31.624097062 +0000 UTC m=+1151.628288902" watchObservedRunningTime="2026-09-17 23:25:31.62652914 +0000 UTC m=+1151.630720979"
Sep 17 23:25:39.457604 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.457522    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"]
Sep 17 23:25:39.461296 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.461278    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:39.465791 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.465763    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-6275e9\""
Sep 17 23:25:39.475795 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.475771    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"]
Sep 17 23:25:39.479695 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.479669    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"]
Sep 17 23:25:39.483190 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.483172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:39.487057 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.487037    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-6275e9-dockercfg-wrs85\""
Sep 17 23:25:39.506401 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.506374    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"]
Sep 17 23:25:39.569641 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.569602    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96c91e78-9d87-4285-aef5-187fdb008c6c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn\" (UID: \"96c91e78-9d87-4285-aef5-187fdb008c6c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:39.569834 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.569756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v\" (UID: \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:39.569912 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.569878    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5ckp\" (UniqueName: \"kubernetes.io/projected/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-kube-api-access-b5ckp\") pod \"payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v\" (UID: \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:39.569972 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.569920    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqw9f\" (UniqueName: \"kubernetes.io/projected/96c91e78-9d87-4285-aef5-187fdb008c6c-kube-api-access-wqw9f\") pod \"payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn\" (UID: \"96c91e78-9d87-4285-aef5-187fdb008c6c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:39.671120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.671082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5ckp\" (UniqueName: \"kubernetes.io/projected/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-kube-api-access-b5ckp\") pod \"payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v\" (UID: \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:39.671120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.671138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqw9f\" (UniqueName: \"kubernetes.io/projected/96c91e78-9d87-4285-aef5-187fdb008c6c-kube-api-access-wqw9f\") pod \"payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn\" (UID: \"96c91e78-9d87-4285-aef5-187fdb008c6c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:39.671383 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.671185    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96c91e78-9d87-4285-aef5-187fdb008c6c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn\" (UID: \"96c91e78-9d87-4285-aef5-187fdb008c6c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:39.671383 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.671305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v\" (UID: \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:39.672055 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.672030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v\" (UID: \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:39.672193 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.672031    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96c91e78-9d87-4285-aef5-187fdb008c6c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn\" (UID: \"96c91e78-9d87-4285-aef5-187fdb008c6c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:39.680690 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.680658    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5ckp\" (UniqueName: \"kubernetes.io/projected/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-kube-api-access-b5ckp\") pod \"payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v\" (UID: \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:39.680801 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.680764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqw9f\" (UniqueName: \"kubernetes.io/projected/96c91e78-9d87-4285-aef5-187fdb008c6c-kube-api-access-wqw9f\") pod \"payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn\" (UID: \"96c91e78-9d87-4285-aef5-187fdb008c6c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:39.773317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.772957    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:39.793721 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:39.793683    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:40.057969 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.057945    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"]
Sep 17 23:25:40.058758 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:25:40.058726    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e2b8d78_3630_4894_a3aa_c96d5fcf9e3c.slice/crio-a17bcb746dc01b73e4d65211e2a48cd91855fe5443350180531b6f2de44d924f WatchSource:0}: Error finding container a17bcb746dc01b73e4d65211e2a48cd91855fe5443350180531b6f2de44d924f: Status 404 returned error can't find the container with id a17bcb746dc01b73e4d65211e2a48cd91855fe5443350180531b6f2de44d924f
Sep 17 23:25:40.086289 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.086264    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"]
Sep 17 23:25:40.094305 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:25:40.094283    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c91e78_9d87_4285_aef5_187fdb008c6c.slice/crio-9d3940bd6d58a36c194c17ae664af116cf8684f358711de75274088fcf86bd62 WatchSource:0}: Error finding container 9d3940bd6d58a36c194c17ae664af116cf8684f358711de75274088fcf86bd62: Status 404 returned error can't find the container with id 9d3940bd6d58a36c194c17ae664af116cf8684f358711de75274088fcf86bd62
Sep 17 23:25:40.658951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.658915    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:40.659428 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.658965    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:40.670205 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.670167    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v" event={"ID":"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c","Type":"ContainerStarted","Data":"bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c"}
Sep 17 23:25:40.670388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.670246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v" event={"ID":"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c","Type":"ContainerStarted","Data":"a17bcb746dc01b73e4d65211e2a48cd91855fe5443350180531b6f2de44d924f"}
Sep 17 23:25:40.670388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.670302    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:40.672370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.672342    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn" event={"ID":"96c91e78-9d87-4285-aef5-187fdb008c6c","Type":"ContainerStarted","Data":"cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd"}
Sep 17 23:25:40.672491 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.672397    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:40.672491 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.672420    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn" event={"ID":"96c91e78-9d87-4285-aef5-187fdb008c6c","Type":"ContainerStarted","Data":"9d3940bd6d58a36c194c17ae664af116cf8684f358711de75274088fcf86bd62"}
Sep 17 23:25:40.674848 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.674827    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:40.692097 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.692027    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v" podStartSLOduration=1.6920080469999998 podStartE2EDuration="1.692008047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:25:40.690212489 +0000 UTC m=+1160.694404328" watchObservedRunningTime="2026-09-17 23:25:40.692008047 +0000 UTC m=+1160.696199891"
Sep 17 23:25:40.728745 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:40.728689    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn" podStartSLOduration=1.728673562 podStartE2EDuration="1.728673562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:25:40.727344229 +0000 UTC m=+1160.731536068" watchObservedRunningTime="2026-09-17 23:25:40.728673562 +0000 UTC m=+1160.732865400"
Sep 17 23:25:41.706069 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:41.706026    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:25:51.685831 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:51.685795    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:25:51.686313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:51.686238    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:25:53.727239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:25:53.727196    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"]
Sep 17 23:26:08.748015 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:08.747963    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs" podUID="b41c8292-07b1-45d8-aa34-26130de9a959" containerName="main" containerID="cri-o://d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41" gracePeriod=45
Sep 17 23:26:09.004957 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.004898    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:26:09.064806 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.064779    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5jjj\" (UniqueName: \"kubernetes.io/projected/b41c8292-07b1-45d8-aa34-26130de9a959-kube-api-access-t5jjj\") pod \"b41c8292-07b1-45d8-aa34-26130de9a959\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") "
Sep 17 23:26:09.064806 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.064808    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-dshm\") pod \"b41c8292-07b1-45d8-aa34-26130de9a959\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") "
Sep 17 23:26:09.065024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.064891    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-model-cache\") pod \"b41c8292-07b1-45d8-aa34-26130de9a959\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") "
Sep 17 23:26:09.065024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.064907    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-tmp-dir\") pod \"b41c8292-07b1-45d8-aa34-26130de9a959\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") "
Sep 17 23:26:09.065024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.064937    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41c8292-07b1-45d8-aa34-26130de9a959-tls-certs\") pod \"b41c8292-07b1-45d8-aa34-26130de9a959\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") "
Sep 17 23:26:09.065024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.064958    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-home\") pod \"b41c8292-07b1-45d8-aa34-26130de9a959\" (UID: \"b41c8292-07b1-45d8-aa34-26130de9a959\") "
Sep 17 23:26:09.065267 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.065193    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-model-cache" (OuterVolumeSpecName: "model-cache") pod "b41c8292-07b1-45d8-aa34-26130de9a959" (UID: "b41c8292-07b1-45d8-aa34-26130de9a959"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:26:09.065267 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.065211    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b41c8292-07b1-45d8-aa34-26130de9a959" (UID: "b41c8292-07b1-45d8-aa34-26130de9a959"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:26:09.065388 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.065358    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-home" (OuterVolumeSpecName: "home") pod "b41c8292-07b1-45d8-aa34-26130de9a959" (UID: "b41c8292-07b1-45d8-aa34-26130de9a959"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:26:09.065830 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.065804    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:26:09.065962 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.065835    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:26:09.065962 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.065850    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:26:09.067255 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.067231    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-dshm" (OuterVolumeSpecName: "dshm") pod "b41c8292-07b1-45d8-aa34-26130de9a959" (UID: "b41c8292-07b1-45d8-aa34-26130de9a959"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:26:09.067530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.067511    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b41c8292-07b1-45d8-aa34-26130de9a959-kube-api-access-t5jjj" (OuterVolumeSpecName: "kube-api-access-t5jjj") pod "b41c8292-07b1-45d8-aa34-26130de9a959" (UID: "b41c8292-07b1-45d8-aa34-26130de9a959"). InnerVolumeSpecName "kube-api-access-t5jjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:26:09.067594 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.067538    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b41c8292-07b1-45d8-aa34-26130de9a959-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b41c8292-07b1-45d8-aa34-26130de9a959" (UID: "b41c8292-07b1-45d8-aa34-26130de9a959"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:26:09.167191 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.167143    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5jjj\" (UniqueName: \"kubernetes.io/projected/b41c8292-07b1-45d8-aa34-26130de9a959-kube-api-access-t5jjj\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:26:09.167191 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.167184    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41c8292-07b1-45d8-aa34-26130de9a959-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:26:09.167191 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.167198    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41c8292-07b1-45d8-aa34-26130de9a959-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:26:09.861336 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.861300    2581 generic.go:356] "Generic (PLEG): container finished" podID="b41c8292-07b1-45d8-aa34-26130de9a959" containerID="d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41" exitCode=0
Sep 17 23:26:09.861784 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.861346    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"
Sep 17 23:26:09.861784 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.861387    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs" event={"ID":"b41c8292-07b1-45d8-aa34-26130de9a959","Type":"ContainerDied","Data":"d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41"}
Sep 17 23:26:09.861784 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.861425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs" event={"ID":"b41c8292-07b1-45d8-aa34-26130de9a959","Type":"ContainerDied","Data":"87561251ca2172b907d92d7875c42c30877ab3c4a21b1fc77ea54215b8fff0ba"}
Sep 17 23:26:09.861784 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.861443    2581 scope.go:117] "RemoveContainer" containerID="d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41"
Sep 17 23:26:09.871944 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.871922    2581 scope.go:117] "RemoveContainer" containerID="d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41"
Sep 17 23:26:09.872223 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:26:09.872206    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41\": container with ID starting with d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41 not found: ID does not exist" containerID="d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41"
Sep 17 23:26:09.872283 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.872232    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41"} err="failed to get container status \"d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41\": rpc error: code = NotFound desc = could not find container \"d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41\": container with ID starting with d13bb883748da13ccac5f9a2a7b78a54154e2b1d5a5b81fc068ab396a018ab41 not found: ID does not exist"
Sep 17 23:26:09.883611 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.883585    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"]
Sep 17 23:26:09.888091 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.888067    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/e2e-conflict-a-21551ab9-kserve-c64594c85-dltcs"]
Sep 17 23:26:09.994629 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:09.994595    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b41c8292-07b1-45d8-aa34-26130de9a959" path="/var/lib/kubelet/pods/b41c8292-07b1-45d8-aa34-26130de9a959/volumes"
Sep 17 23:26:22.246063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:22.246029    2581 scope.go:117] "RemoveContainer" containerID="644a1c626bf199ef6e6d54c52aabd3eca2c1365869930768f280f00732c98c9f"
Sep 17 23:26:22.255026 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:22.255005    2581 scope.go:117] "RemoveContainer" containerID="b3a5d2c685284c94686c1b4dcc82bef4c60bcc275609c581818f3598e83270c5"
Sep 17 23:26:46.407357 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.407318    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"]
Sep 17 23:26:46.407997 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.407976    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b41c8292-07b1-45d8-aa34-26130de9a959" containerName="main"
Sep 17 23:26:46.408089 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.408000    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b41c8292-07b1-45d8-aa34-26130de9a959" containerName="main"
Sep 17 23:26:46.408195 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.408181    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b41c8292-07b1-45d8-aa34-26130de9a959" containerName="main"
Sep 17 23:26:46.411318 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.411298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.427531 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:26:46.427503    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"auth-test-model-e5aae7-kserve-self-signed-certs\" is forbidden: User \"system:node:ip-10-0-128-233.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-shared-a-w3-e5aae7\": no relationship found between node 'ip-10-0-128-233.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-shared-a-w3-e5aae7\"/\"auth-test-model-e5aae7-kserve-self-signed-certs\"" type="*v1.Secret"
Sep 17 23:26:46.428349 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:26:46.428312    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-plbq2\" is forbidden: User \"system:node:ip-10-0-128-233.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-shared-a-w3-e5aae7\": no relationship found between node 'ip-10-0-128-233.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-shared-a-w3-e5aae7\"/\"default-dockercfg-plbq2\"" type="*v1.Secret"
Sep 17 23:26:46.428538 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:26:46.428432    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-233.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-shared-a-w3-e5aae7\": no relationship found between node 'ip-10-0-128-233.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-shared-a-w3-e5aae7\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 17 23:26:46.428761 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:26:46.428736    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" is forbidden: User \"system:node:ip-10-0-128-233.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-shared-a-w3-e5aae7\": no relationship found between node 'ip-10-0-128-233.ec2.internal' and this object" podUID="33b3ce28-cfaf-4622-871e-97e01dd54f9f" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.428883 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:26:46.428865    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-233.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-shared-a-w3-e5aae7\": no relationship found between node 'ip-10-0-128-233.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-shared-a-w3-e5aae7\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 17 23:26:46.454190 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.454159    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"]
Sep 17 23:26:46.538667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.538630    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-dshm\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.538667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.538663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tls-certs\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.538877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.538689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tmp-dir\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.538877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.538767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-home\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.538877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.538794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-model-cache\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.538877 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.538862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9f9z\" (UniqueName: \"kubernetes.io/projected/33b3ce28-cfaf-4622-871e-97e01dd54f9f-kube-api-access-h9f9z\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.639968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.639918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-home\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.640169 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.639978    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-model-cache\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.640169 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.640033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9f9z\" (UniqueName: \"kubernetes.io/projected/33b3ce28-cfaf-4622-871e-97e01dd54f9f-kube-api-access-h9f9z\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.640169 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.640096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-dshm\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.640169 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.640115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tls-certs\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.640377 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.640174    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tmp-dir\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.640524 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.640501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-home\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.640560 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.640540    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-model-cache\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.640595 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.640576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tmp-dir\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:46.642435 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:46.642415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-dshm\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:47.241943 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:47.241904    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-e5aae7\"/\"openshift-service-ca.crt\""
Sep 17 23:26:47.300910 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:47.300876    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-e5aae7\"/\"auth-test-model-e5aae7-kserve-self-signed-certs\""
Sep 17 23:26:47.313822 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:47.313788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tls-certs\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:47.446235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:47.446202    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-e5aae7\"/\"kube-root-ca.crt\""
Sep 17 23:26:47.450566 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:47.450541    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9f9z\" (UniqueName: \"kubernetes.io/projected/33b3ce28-cfaf-4622-871e-97e01dd54f9f-kube-api-access-h9f9z\") pod \"auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") " pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:47.703183 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:47.703155    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-e5aae7\"/\"default-dockercfg-plbq2\""
Sep 17 23:26:47.711330 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:47.711309    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:47.984918 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:47.984890    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"]
Sep 17 23:26:47.985571 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:26:47.985546    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33b3ce28_cfaf_4622_871e_97e01dd54f9f.slice/crio-f3b8fcf1c6f2badc918ee5c18f50d21b842c65b90620182a127f51d4f465ebb2 WatchSource:0}: Error finding container f3b8fcf1c6f2badc918ee5c18f50d21b842c65b90620182a127f51d4f465ebb2: Status 404 returned error can't find the container with id f3b8fcf1c6f2badc918ee5c18f50d21b842c65b90620182a127f51d4f465ebb2
Sep 17 23:26:48.103255 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:48.103221    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4" event={"ID":"33b3ce28-cfaf-4622-871e-97e01dd54f9f","Type":"ContainerStarted","Data":"432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f"}
Sep 17 23:26:48.103418 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:48.103261    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4" event={"ID":"33b3ce28-cfaf-4622-871e-97e01dd54f9f","Type":"ContainerStarted","Data":"f3b8fcf1c6f2badc918ee5c18f50d21b842c65b90620182a127f51d4f465ebb2"}
Sep 17 23:26:48.121824 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:48.121768    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4" podStartSLOduration=2.121751367 podStartE2EDuration="2.121751367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:26:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:26:48.120337417 +0000 UTC m=+1228.124529255" watchObservedRunningTime="2026-09-17 23:26:48.121751367 +0000 UTC m=+1228.125943236"
Sep 17 23:26:57.712038 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:57.711999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:57.712540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:57.712053    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:57.724668 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:57.724641    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:26:58.178696 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:26:58.178665    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:27:17.175453 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.175420    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"]
Sep 17 23:27:17.175935 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.175619    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2" podUID="c166e699-5ca2-4747-8590-e9a7511f2a4f" containerName="payload-pre-processing" containerID="cri-o://e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4" gracePeriod=30
Sep 17 23:27:17.452896 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.452870    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:27:17.577720 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.577680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2nq2\" (UniqueName: \"kubernetes.io/projected/c166e699-5ca2-4747-8590-e9a7511f2a4f-kube-api-access-f2nq2\") pod \"c166e699-5ca2-4747-8590-e9a7511f2a4f\" (UID: \"c166e699-5ca2-4747-8590-e9a7511f2a4f\") "
Sep 17 23:27:17.577929 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.577786    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c166e699-5ca2-4747-8590-e9a7511f2a4f-plugins-config-volume\") pod \"c166e699-5ca2-4747-8590-e9a7511f2a4f\" (UID: \"c166e699-5ca2-4747-8590-e9a7511f2a4f\") "
Sep 17 23:27:17.578203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.578174    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c166e699-5ca2-4747-8590-e9a7511f2a4f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c166e699-5ca2-4747-8590-e9a7511f2a4f" (UID: "c166e699-5ca2-4747-8590-e9a7511f2a4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:27:17.579857 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.579833    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c166e699-5ca2-4747-8590-e9a7511f2a4f-kube-api-access-f2nq2" (OuterVolumeSpecName: "kube-api-access-f2nq2") pod "c166e699-5ca2-4747-8590-e9a7511f2a4f" (UID: "c166e699-5ca2-4747-8590-e9a7511f2a4f"). InnerVolumeSpecName "kube-api-access-f2nq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:27:17.679740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.679666    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2nq2\" (UniqueName: \"kubernetes.io/projected/c166e699-5ca2-4747-8590-e9a7511f2a4f-kube-api-access-f2nq2\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:27:17.679740 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:17.679694    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c166e699-5ca2-4747-8590-e9a7511f2a4f-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:27:18.293398 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.293303    2581 generic.go:356] "Generic (PLEG): container finished" podID="c166e699-5ca2-4747-8590-e9a7511f2a4f" containerID="e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4" exitCode=0
Sep 17 23:27:18.293398 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.293357    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"
Sep 17 23:27:18.293862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.293396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2" event={"ID":"c166e699-5ca2-4747-8590-e9a7511f2a4f","Type":"ContainerDied","Data":"e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4"}
Sep 17 23:27:18.293862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.293475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2" event={"ID":"c166e699-5ca2-4747-8590-e9a7511f2a4f","Type":"ContainerDied","Data":"875c34336b898e7fdece52008d18ddc4b382fe7c8567d0e9630ccc437dbc7a47"}
Sep 17 23:27:18.293862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.293500    2581 scope.go:117] "RemoveContainer" containerID="e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4"
Sep 17 23:27:18.303235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.303217    2581 scope.go:117] "RemoveContainer" containerID="e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4"
Sep 17 23:27:18.303535 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:27:18.303513    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4\": container with ID starting with e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4 not found: ID does not exist" containerID="e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4"
Sep 17 23:27:18.303611 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.303543    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4"} err="failed to get container status \"e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4\": rpc error: code = NotFound desc = could not find container \"e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4\": container with ID starting with e487c5ed0d9af679854b49059d33f813e815a1e820ae3a772e8b4af7e54e39e4 not found: ID does not exist"
Sep 17 23:27:18.315737 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.315712    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"]
Sep 17 23:27:18.318491 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:18.318466    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6824f488-787c5fd66-nx9f2"]
Sep 17 23:27:19.994560 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:27:19.994530    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c166e699-5ca2-4747-8590-e9a7511f2a4f" path="/var/lib/kubelet/pods/c166e699-5ca2-4747-8590-e9a7511f2a4f/volumes"
Sep 17 23:28:14.403737 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:14.403694    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"]
Sep 17 23:28:14.404313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:14.403976    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy" containerID="cri-o://430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f" gracePeriod=30
Sep 17 23:28:17.160812 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.160772    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"]
Sep 17 23:28:17.161331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.161251    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c166e699-5ca2-4747-8590-e9a7511f2a4f" containerName="payload-pre-processing"
Sep 17 23:28:17.161331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.161264    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c166e699-5ca2-4747-8590-e9a7511f2a4f" containerName="payload-pre-processing"
Sep 17 23:28:17.161468 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.161335    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c166e699-5ca2-4747-8590-e9a7511f2a4f" containerName="payload-pre-processing"
Sep 17 23:28:17.164317 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.164298    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.167086 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.167058    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-0b4d93\"/\"embedding-w0-0b4d93-kserve-self-signed-certs\""
Sep 17 23:28:17.179039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.179011    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"]
Sep 17 23:28:17.280709 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.280670    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-model-cache\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.280709 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.280719    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg5lr\" (UniqueName: \"kubernetes.io/projected/d4ec9362-ae8d-429d-9275-06fa40a9263d-kube-api-access-pg5lr\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.280999 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.280764    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-home\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.280999 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.280922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-tmp-dir\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.281142 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.281055    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-dshm\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.281142 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.281094    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ec9362-ae8d-429d-9275-06fa40a9263d-tls-certs\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.330032 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.329996    2581 patch_prober.go:28] interesting pod/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 23:28:17.330216 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.330065    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 23:28:17.382174 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-home\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.382364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382265    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-tmp-dir\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.382446 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-dshm\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.382446 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382391    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ec9362-ae8d-429d-9275-06fa40a9263d-tls-certs\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.382446 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-model-cache\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.382620 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg5lr\" (UniqueName: \"kubernetes.io/projected/d4ec9362-ae8d-429d-9275-06fa40a9263d-kube-api-access-pg5lr\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.382620 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382572    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-home\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.382620 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382581    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-tmp-dir\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.382851 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.382829    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-model-cache\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.385180 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.385119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-dshm\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.385703 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.385680    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ec9362-ae8d-429d-9275-06fa40a9263d-tls-certs\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.399116 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.399087    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg5lr\" (UniqueName: \"kubernetes.io/projected/d4ec9362-ae8d-429d-9275-06fa40a9263d-kube-api-access-pg5lr\") pod \"embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") " pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.474802 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.474665    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:17.749470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:17.749441    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"]
Sep 17 23:28:17.750491 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:28:17.750466    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4ec9362_ae8d_429d_9275_06fa40a9263d.slice/crio-f465a42a87b90885160d6749546bc450449a61c1824edc83e445d726a0a25174 WatchSource:0}: Error finding container f465a42a87b90885160d6749546bc450449a61c1824edc83e445d726a0a25174: Status 404 returned error can't find the container with id f465a42a87b90885160d6749546bc450449a61c1824edc83e445d726a0a25174
Sep 17 23:28:18.667572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:18.667534    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf" event={"ID":"d4ec9362-ae8d-429d-9275-06fa40a9263d","Type":"ContainerStarted","Data":"680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2"}
Sep 17 23:28:18.667971 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:18.667586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf" event={"ID":"d4ec9362-ae8d-429d-9275-06fa40a9263d","Type":"ContainerStarted","Data":"f465a42a87b90885160d6749546bc450449a61c1824edc83e445d726a0a25174"}
Sep 17 23:28:18.688616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:18.688565    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf" podStartSLOduration=1.688550678 podStartE2EDuration="1.688550678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:28:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:28:18.685932733 +0000 UTC m=+1318.690124574" watchObservedRunningTime="2026-09-17 23:28:18.688550678 +0000 UTC m=+1318.692742517"
Sep 17 23:28:19.661294 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.661270    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:28:19.674420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.674352    2581 generic.go:356] "Generic (PLEG): container finished" podID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerID="430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f" exitCode=0
Sep 17 23:28:19.674420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.674395    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"
Sep 17 23:28:19.674777 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.674440    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" event={"ID":"15d2a174-f8d5-4f5e-9e65-a0919f106b69","Type":"ContainerDied","Data":"430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f"}
Sep 17 23:28:19.674777 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.674489    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x" event={"ID":"15d2a174-f8d5-4f5e-9e65-a0919f106b69","Type":"ContainerDied","Data":"3d42644ce37b9a5f023351c364579de34111a2ae607c5c1fb4b5117973d7e2d8"}
Sep 17 23:28:19.674777 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.674513    2581 scope.go:117] "RemoveContainer" containerID="430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f"
Sep 17 23:28:19.685425 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.685408    2581 scope.go:117] "RemoveContainer" containerID="9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588"
Sep 17 23:28:19.696323 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.696302    2581 scope.go:117] "RemoveContainer" containerID="430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f"
Sep 17 23:28:19.696668 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:28:19.696646    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f\": container with ID starting with 430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f not found: ID does not exist" containerID="430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f"
Sep 17 23:28:19.696758 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.696681    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f"} err="failed to get container status \"430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f\": rpc error: code = NotFound desc = could not find container \"430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f\": container with ID starting with 430192cc37d4dc1faff892cb5d1a4a1876f25cd624861460e26db6023490b52f not found: ID does not exist"
Sep 17 23:28:19.696758 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.696705    2581 scope.go:117] "RemoveContainer" containerID="9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588"
Sep 17 23:28:19.698651 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:28:19.698623    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588\": container with ID starting with 9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588 not found: ID does not exist" containerID="9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588"
Sep 17 23:28:19.698854 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.698824    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588"} err="failed to get container status \"9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588\": rpc error: code = NotFound desc = could not find container \"9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588\": container with ID starting with 9126029bd288ae3368ade48b71acaf23f8c1fdb9a3d57fa3c32e30586aae3588 not found: ID does not exist"
Sep 17 23:28:19.704404 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704381    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbgpt\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-kube-api-access-jbgpt\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704501 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704453    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-certs\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704547 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704514    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istiod-ca-cert\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704547 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704538    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-envoy\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704689 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704580    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-data\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704689 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-podinfo\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704689 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704679    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-socket\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704851 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704723    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-credential-socket\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704851 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704766    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-token\") pod \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\" (UID: \"15d2a174-f8d5-4f5e-9e65-a0919f106b69\") "
Sep 17 23:28:19.704851 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.704807    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:19.705032 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.705011    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:19.705092 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.705052    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:28:19.705357 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.705270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-data" (OuterVolumeSpecName: "istio-data") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:19.705357 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.705304    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-credential-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.705357 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.705320    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:19.705357 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.705340    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.705601 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.705380    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istiod-ca-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.707392 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.707367    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-token" (OuterVolumeSpecName: "istio-token") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:28:19.707392 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.707383    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:28:19.707639 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.707612    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:28:19.707906 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.707883    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-kube-api-access-jbgpt" (OuterVolumeSpecName: "kube-api-access-jbgpt") pod "15d2a174-f8d5-4f5e-9e65-a0919f106b69" (UID: "15d2a174-f8d5-4f5e-9e65-a0919f106b69"). InnerVolumeSpecName "kube-api-access-jbgpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:28:19.806664 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.806633    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-envoy\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.806664 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.806657    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-data\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.806664 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.806667    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-podinfo\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.806980 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.806677    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d2a174-f8d5-4f5e-9e65-a0919f106b69-workload-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.806980 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.806686    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-istio-token\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.806980 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.806695    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbgpt\" (UniqueName: \"kubernetes.io/projected/15d2a174-f8d5-4f5e-9e65-a0919f106b69-kube-api-access-jbgpt\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:28:19.998819 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:19.998782    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"]
Sep 17 23:28:20.002772 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:20.002742    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6824f488-openshift-default-846cb44784-qxw7x"]
Sep 17 23:28:21.995470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:21.995439    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" path="/var/lib/kubelet/pods/15d2a174-f8d5-4f5e-9e65-a0919f106b69/volumes"
Sep 17 23:28:22.298766 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:22.298682    2581 scope.go:117] "RemoveContainer" containerID="d686b91ba25496f6df56845e60dc1ef2355ef793562e193ab2313777780aaf3e"
Sep 17 23:28:27.475005 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:27.474974    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:27.475419 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:27.475315    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:27.487418 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:27.487394    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:28:27.736205 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:28:27.736110    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:29:16.720750 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.720715    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"]
Sep 17 23:29:16.721211 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.721186    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy"
Sep 17 23:29:16.721211 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.721211    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy"
Sep 17 23:29:16.721211 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.721219    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy"
Sep 17 23:29:16.721331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.721225    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy"
Sep 17 23:29:16.721331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.721298    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy"
Sep 17 23:29:16.721331 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.721307    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d2a174-f8d5-4f5e-9e65-a0919f106b69" containerName="istio-proxy"
Sep 17 23:29:16.724363 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.724347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:16.726748 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.726726    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-eb0661\""
Sep 17 23:29:16.726873 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.726750    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-eb0661-dockercfg-llj6p\""
Sep 17 23:29:16.737955 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.737925    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"]
Sep 17 23:29:16.759539 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.759516    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"]
Sep 17 23:29:16.763136 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.763106    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:16.783694 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.783662    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"]
Sep 17 23:29:16.875041 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.875011    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sp95\" (UniqueName: \"kubernetes.io/projected/bb663080-1e12-45c1-af0a-02671cafc675-kube-api-access-5sp95\") pod \"payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9\" (UID: \"bb663080-1e12-45c1-af0a-02671cafc675\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:16.875281 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.875070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjb4d\" (UniqueName: \"kubernetes.io/projected/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-kube-api-access-rjb4d\") pod \"payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb\" (UID: \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:16.875281 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.875193    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb663080-1e12-45c1-af0a-02671cafc675-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9\" (UID: \"bb663080-1e12-45c1-af0a-02671cafc675\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:16.875281 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.875253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb\" (UID: \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:16.976165 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.976068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb663080-1e12-45c1-af0a-02671cafc675-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9\" (UID: \"bb663080-1e12-45c1-af0a-02671cafc675\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:16.976165 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.976115    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb\" (UID: \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:16.976398 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.976222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sp95\" (UniqueName: \"kubernetes.io/projected/bb663080-1e12-45c1-af0a-02671cafc675-kube-api-access-5sp95\") pod \"payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9\" (UID: \"bb663080-1e12-45c1-af0a-02671cafc675\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:16.976398 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.976279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjb4d\" (UniqueName: \"kubernetes.io/projected/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-kube-api-access-rjb4d\") pod \"payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb\" (UID: \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:16.976892 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.976866    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb663080-1e12-45c1-af0a-02671cafc675-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9\" (UID: \"bb663080-1e12-45c1-af0a-02671cafc675\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:16.976983 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.976946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb\" (UID: \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:16.984633 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.984609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sp95\" (UniqueName: \"kubernetes.io/projected/bb663080-1e12-45c1-af0a-02671cafc675-kube-api-access-5sp95\") pod \"payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9\" (UID: \"bb663080-1e12-45c1-af0a-02671cafc675\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:16.984731 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:16.984705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjb4d\" (UniqueName: \"kubernetes.io/projected/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-kube-api-access-rjb4d\") pod \"payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb\" (UID: \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\") " pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:17.035341 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:17.035311    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:17.074176 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:17.074107    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:17.313377 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:17.313348    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"]
Sep 17 23:29:17.313966 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:29:17.313943    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb663080_1e12_45c1_af0a_02671cafc675.slice/crio-708984d4dbd4e54cc244149cd82e81e64005c72e1aa689ca42ae7a75d9cdc7ea WatchSource:0}: Error finding container 708984d4dbd4e54cc244149cd82e81e64005c72e1aa689ca42ae7a75d9cdc7ea: Status 404 returned error can't find the container with id 708984d4dbd4e54cc244149cd82e81e64005c72e1aa689ca42ae7a75d9cdc7ea
Sep 17 23:29:17.377971 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:17.377936    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"]
Sep 17 23:29:17.379375 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:29:17.379351    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ae9e6f_454c_428a_b2cb_7d65f15d18fd.slice/crio-e1367d6e254da5942b95dccb380942b7fb656108f3cfedf4f5561e2ef1e596a0 WatchSource:0}: Error finding container e1367d6e254da5942b95dccb380942b7fb656108f3cfedf4f5561e2ef1e596a0: Status 404 returned error can't find the container with id e1367d6e254da5942b95dccb380942b7fb656108f3cfedf4f5561e2ef1e596a0
Sep 17 23:29:18.043194 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:18.043158    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb" event={"ID":"73ae9e6f-454c-428a-b2cb-7d65f15d18fd","Type":"ContainerStarted","Data":"66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90"}
Sep 17 23:29:18.043661 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:18.043217    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb" event={"ID":"73ae9e6f-454c-428a-b2cb-7d65f15d18fd","Type":"ContainerStarted","Data":"e1367d6e254da5942b95dccb380942b7fb656108f3cfedf4f5561e2ef1e596a0"}
Sep 17 23:29:18.043661 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:18.043346    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:18.044806 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:18.044786    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9" event={"ID":"bb663080-1e12-45c1-af0a-02671cafc675","Type":"ContainerStarted","Data":"fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4"}
Sep 17 23:29:18.044873 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:18.044826    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:18.044873 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:18.044842    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9" event={"ID":"bb663080-1e12-45c1-af0a-02671cafc675","Type":"ContainerStarted","Data":"708984d4dbd4e54cc244149cd82e81e64005c72e1aa689ca42ae7a75d9cdc7ea"}
Sep 17 23:29:18.062927 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:18.062881    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb" podStartSLOduration=2.062868564 podStartE2EDuration="2.062868564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:29:18.060887845 +0000 UTC m=+1378.065079687" watchObservedRunningTime="2026-09-17 23:29:18.062868564 +0000 UTC m=+1378.067060409"
Sep 17 23:29:18.078366 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:18.078322    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9" podStartSLOduration=2.078308358 podStartE2EDuration="2.078308358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:29:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:29:18.076091628 +0000 UTC m=+1378.080283467" watchObservedRunningTime="2026-09-17 23:29:18.078308358 +0000 UTC m=+1378.082500262"
Sep 17 23:29:22.324654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:22.324618    2581 scope.go:117] "RemoveContainer" containerID="cf46a71dc1537c2dbd6989c730bdacf1616c9530e3be067a0bf03b9c986adbb1"
Sep 17 23:29:29.072843 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:29.072809    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:29:29.073235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:29.072987    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:29:51.387744 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:51.387702    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"]
Sep 17 23:29:51.402120 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:29:51.402089    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"]
Sep 17 23:30:00.150280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.150241    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"]
Sep 17 23:30:00.159204 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.157671    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"]
Sep 17 23:30:00.159204 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.159068    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"
Sep 17 23:30:00.163369 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.163342    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"]
Sep 17 23:30:00.163518 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.163466    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"
Sep 17 23:30:00.167352 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.167061    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"
Sep 17 23:30:00.167484 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.167466    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:30:00.171181 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.171162    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"
Sep 17 23:30:00.182876 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.182855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"
Sep 17 23:30:00.214588 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.214503    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"]
Sep 17 23:30:00.217499 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.217110    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"]
Sep 17 23:30:00.221229 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.221173    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"]
Sep 17 23:30:00.226046 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.226024    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"
Sep 17 23:30:00.250188 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.248636    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk"]
Sep 17 23:30:00.255166 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.253946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk"
Sep 17 23:30:00.265934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.265348    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"]
Sep 17 23:30:00.277262 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.276376    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"
Sep 17 23:30:00.277863 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.277790    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk"]
Sep 17 23:30:00.287932 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.287911    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk"
Sep 17 23:30:00.298500 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.298475    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"]
Sep 17 23:30:00.300052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.299889    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"
Sep 17 23:30:00.467359 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.467327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"]
Sep 17 23:30:00.468933 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:30:00.468910    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25cf16e7_a51b_4e40_aac3_6060e94cc265.slice/crio-fe92db77cd67562d42f65e345628199fa143b83262240dffc707e7dbcaaca3bb WatchSource:0}: Error finding container fe92db77cd67562d42f65e345628199fa143b83262240dffc707e7dbcaaca3bb: Status 404 returned error can't find the container with id fe92db77cd67562d42f65e345628199fa143b83262240dffc707e7dbcaaca3bb
Sep 17 23:30:00.481792 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.481772    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"]
Sep 17 23:30:00.484241 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:30:00.484216    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod518a9c6d_776e_4289_8a0a_dca6b1ad17db.slice/crio-3fe2a7049e14e73feaf85edfdbe914ac7294f0d5fadb743ac52d3c01ce6d92cf WatchSource:0}: Error finding container 3fe2a7049e14e73feaf85edfdbe914ac7294f0d5fadb743ac52d3c01ce6d92cf: Status 404 returned error can't find the container with id 3fe2a7049e14e73feaf85edfdbe914ac7294f0d5fadb743ac52d3c01ce6d92cf
Sep 17 23:30:00.538702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.538676    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"]
Sep 17 23:30:00.539056 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:30:00.539034    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7090575f_4f2e_493f_8c13_45005510eb90.slice/crio-285effa30fbbf00788a50e07a0f1b8a9ce4838bf87ba3417252d55785e2e9d86 WatchSource:0}: Error finding container 285effa30fbbf00788a50e07a0f1b8a9ce4838bf87ba3417252d55785e2e9d86: Status 404 returned error can't find the container with id 285effa30fbbf00788a50e07a0f1b8a9ce4838bf87ba3417252d55785e2e9d86
Sep 17 23:30:00.593379 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.593349    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk"]
Sep 17 23:30:00.594599 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:30:00.594577    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2736b23a_1440_4d3f_b591_a519b8842554.slice/crio-52565b61eaad1778135931b0c317a4846016b44e7e9483d7239c5a58a3261020 WatchSource:0}: Error finding container 52565b61eaad1778135931b0c317a4846016b44e7e9483d7239c5a58a3261020: Status 404 returned error can't find the container with id 52565b61eaad1778135931b0c317a4846016b44e7e9483d7239c5a58a3261020
Sep 17 23:30:00.609947 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:00.609924    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"]
Sep 17 23:30:00.610941 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:30:00.610918    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf379f212_b36f_4b2f_9595_e6d5163f1dbd.slice/crio-5a312286118cbc67609693bda08e3b895f5c28fd8f3a9431a6f64d3161ea729b WatchSource:0}: Error finding container 5a312286118cbc67609693bda08e3b895f5c28fd8f3a9431a6f64d3161ea729b: Status 404 returned error can't find the container with id 5a312286118cbc67609693bda08e3b895f5c28fd8f3a9431a6f64d3161ea729b
Sep 17 23:30:01.333969 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.333931    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq" event={"ID":"7090575f-4f2e-493f-8c13-45005510eb90","Type":"ContainerStarted","Data":"411075d30ba93cb52ca96af617e3c70d43326c7bef3f2ceb5f55250fc4d01a11"}
Sep 17 23:30:01.334450 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.333996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq" event={"ID":"7090575f-4f2e-493f-8c13-45005510eb90","Type":"ContainerStarted","Data":"285effa30fbbf00788a50e07a0f1b8a9ce4838bf87ba3417252d55785e2e9d86"}
Sep 17 23:30:01.335672 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.335648    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7" event={"ID":"25cf16e7-a51b-4e40-aac3-6060e94cc265","Type":"ContainerStarted","Data":"9b32f96be1bd003e8a985927686b6fc53d62639ed6932dd7b83f593761b2ca0c"}
Sep 17 23:30:01.335795 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.335683    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7" event={"ID":"25cf16e7-a51b-4e40-aac3-6060e94cc265","Type":"ContainerStarted","Data":"fe92db77cd67562d42f65e345628199fa143b83262240dffc707e7dbcaaca3bb"}
Sep 17 23:30:01.337053 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.337030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq" event={"ID":"f379f212-b36f-4b2f-9595-e6d5163f1dbd","Type":"ContainerStarted","Data":"9574dd9d1866f622868cba4375dd98c48dc91a991878af2d6a95c6e3f9687ef4"}
Sep 17 23:30:01.337183 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.337072    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq" event={"ID":"f379f212-b36f-4b2f-9595-e6d5163f1dbd","Type":"ContainerStarted","Data":"5a312286118cbc67609693bda08e3b895f5c28fd8f3a9431a6f64d3161ea729b"}
Sep 17 23:30:01.338536 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.338511    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn" event={"ID":"518a9c6d-776e-4289-8a0a-dca6b1ad17db","Type":"ContainerStarted","Data":"a93f09b92fe918a3d79554a85c4fe857db9110b032eff09461c08d6605f10790"}
Sep 17 23:30:01.338657 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.338554    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn" event={"ID":"518a9c6d-776e-4289-8a0a-dca6b1ad17db","Type":"ContainerStarted","Data":"3fe2a7049e14e73feaf85edfdbe914ac7294f0d5fadb743ac52d3c01ce6d92cf"}
Sep 17 23:30:01.339973 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.339953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk" event={"ID":"2736b23a-1440-4d3f-b591-a519b8842554","Type":"ContainerStarted","Data":"3cd7226d2acd175218c8f57717ff33d66f6da86ce81d2cb593facbb38ec47dee"}
Sep 17 23:30:01.340048 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.339988    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk" event={"ID":"2736b23a-1440-4d3f-b591-a519b8842554","Type":"ContainerStarted","Data":"52565b61eaad1778135931b0c317a4846016b44e7e9483d7239c5a58a3261020"}
Sep 17 23:30:01.350582 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.350544    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq" podStartSLOduration=1.3505296 podStartE2EDuration="1.3505296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:30:01.349450726 +0000 UTC m=+1421.353642568" watchObservedRunningTime="2026-09-17 23:30:01.3505296 +0000 UTC m=+1421.354721439"
Sep 17 23:30:01.365908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.365858    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn" podStartSLOduration=1.365842857 podStartE2EDuration="1.365842857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:30:01.364082184 +0000 UTC m=+1421.368274023" watchObservedRunningTime="2026-09-17 23:30:01.365842857 +0000 UTC m=+1421.370034735"
Sep 17 23:30:01.381178 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.381113    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq" podStartSLOduration=1.381098116 podStartE2EDuration="1.381098116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:30:01.378631789 +0000 UTC m=+1421.382823629" watchObservedRunningTime="2026-09-17 23:30:01.381098116 +0000 UTC m=+1421.385289956"
Sep 17 23:30:01.393701 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.393658    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk" podStartSLOduration=1.393645545 podStartE2EDuration="1.393645545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:30:01.392594486 +0000 UTC m=+1421.396786326" watchObservedRunningTime="2026-09-17 23:30:01.393645545 +0000 UTC m=+1421.397837383"
Sep 17 23:30:01.411864 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:01.411815    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7" podStartSLOduration=1.411797792 podStartE2EDuration="1.411797792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:30:01.408533864 +0000 UTC m=+1421.412725704" watchObservedRunningTime="2026-09-17 23:30:01.411797792 +0000 UTC m=+1421.415989630"
Sep 17 23:30:02.348571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.348527    2581 generic.go:356] "Generic (PLEG): container finished" podID="f379f212-b36f-4b2f-9595-e6d5163f1dbd" containerID="9574dd9d1866f622868cba4375dd98c48dc91a991878af2d6a95c6e3f9687ef4" exitCode=0
Sep 17 23:30:02.349040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.348611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq" event={"ID":"f379f212-b36f-4b2f-9595-e6d5163f1dbd","Type":"ContainerDied","Data":"9574dd9d1866f622868cba4375dd98c48dc91a991878af2d6a95c6e3f9687ef4"}
Sep 17 23:30:02.351544 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.351517    2581 generic.go:356] "Generic (PLEG): container finished" podID="518a9c6d-776e-4289-8a0a-dca6b1ad17db" containerID="a93f09b92fe918a3d79554a85c4fe857db9110b032eff09461c08d6605f10790" exitCode=0
Sep 17 23:30:02.351668 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.351627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn" event={"ID":"518a9c6d-776e-4289-8a0a-dca6b1ad17db","Type":"ContainerDied","Data":"a93f09b92fe918a3d79554a85c4fe857db9110b032eff09461c08d6605f10790"}
Sep 17 23:30:02.353633 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.353611    2581 generic.go:356] "Generic (PLEG): container finished" podID="2736b23a-1440-4d3f-b591-a519b8842554" containerID="3cd7226d2acd175218c8f57717ff33d66f6da86ce81d2cb593facbb38ec47dee" exitCode=0
Sep 17 23:30:02.353753 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.353697    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk" event={"ID":"2736b23a-1440-4d3f-b591-a519b8842554","Type":"ContainerDied","Data":"3cd7226d2acd175218c8f57717ff33d66f6da86ce81d2cb593facbb38ec47dee"}
Sep 17 23:30:02.355551 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.355531    2581 generic.go:356] "Generic (PLEG): container finished" podID="7090575f-4f2e-493f-8c13-45005510eb90" containerID="411075d30ba93cb52ca96af617e3c70d43326c7bef3f2ceb5f55250fc4d01a11" exitCode=7
Sep 17 23:30:02.355669 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.355625    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq" event={"ID":"7090575f-4f2e-493f-8c13-45005510eb90","Type":"ContainerDied","Data":"411075d30ba93cb52ca96af617e3c70d43326c7bef3f2ceb5f55250fc4d01a11"}
Sep 17 23:30:02.355797 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.355780    2581 scope.go:117] "RemoveContainer" containerID="411075d30ba93cb52ca96af617e3c70d43326c7bef3f2ceb5f55250fc4d01a11"
Sep 17 23:30:02.357518 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.357496    2581 generic.go:356] "Generic (PLEG): container finished" podID="25cf16e7-a51b-4e40-aac3-6060e94cc265" containerID="9b32f96be1bd003e8a985927686b6fc53d62639ed6932dd7b83f593761b2ca0c" exitCode=0
Sep 17 23:30:02.357604 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.357557    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7" event={"ID":"25cf16e7-a51b-4e40-aac3-6060e94cc265","Type":"ContainerDied","Data":"9b32f96be1bd003e8a985927686b6fc53d62639ed6932dd7b83f593761b2ca0c"}
Sep 17 23:30:02.690633 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.690599    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"]
Sep 17 23:30:02.690867 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.690840    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd" podUID="9b4b9fa2-6693-4be0-bc8b-c639d5f3191d" containerName="payload-pre-processing" containerID="cri-o://5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801" gracePeriod=30
Sep 17 23:30:02.998328 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:02.998306    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:30:03.050492 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.050458    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4glwz\" (UniqueName: \"kubernetes.io/projected/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-kube-api-access-4glwz\") pod \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\" (UID: \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\") "
Sep 17 23:30:03.050659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.050518    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-plugins-config-volume\") pod \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\" (UID: \"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d\") "
Sep 17 23:30:03.051018 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.050991    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b4b9fa2-6693-4be0-bc8b-c639d5f3191d" (UID: "9b4b9fa2-6693-4be0-bc8b-c639d5f3191d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:30:03.052703 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.052679    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-kube-api-access-4glwz" (OuterVolumeSpecName: "kube-api-access-4glwz") pod "9b4b9fa2-6693-4be0-bc8b-c639d5f3191d" (UID: "9b4b9fa2-6693-4be0-bc8b-c639d5f3191d"). InnerVolumeSpecName "kube-api-access-4glwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:30:03.152347 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.152303    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4glwz\" (UniqueName: \"kubernetes.io/projected/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-kube-api-access-4glwz\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:03.152347 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.152342    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:03.367105 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.367066    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq" event={"ID":"7090575f-4f2e-493f-8c13-45005510eb90","Type":"ContainerStarted","Data":"d71c26cb2fee89121c3da1d9cf3d8db41c08da410c22d4a15095871d4ec6813d"}
Sep 17 23:30:03.368384 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.368355    2581 generic.go:356] "Generic (PLEG): container finished" podID="9b4b9fa2-6693-4be0-bc8b-c639d5f3191d" containerID="5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801" exitCode=0
Sep 17 23:30:03.368511 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.368392    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"
Sep 17 23:30:03.368511 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.368433    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd" event={"ID":"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d","Type":"ContainerDied","Data":"5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801"}
Sep 17 23:30:03.368511 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.368478    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd" event={"ID":"9b4b9fa2-6693-4be0-bc8b-c639d5f3191d","Type":"ContainerDied","Data":"87850c1a45736b6f986d4ae55d5f9c4c2c482858b6e62f919af45fa255136489"}
Sep 17 23:30:03.368511 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.368502    2581 scope.go:117] "RemoveContainer" containerID="5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801"
Sep 17 23:30:03.419379 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.419275    2581 scope.go:117] "RemoveContainer" containerID="5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801"
Sep 17 23:30:03.419657 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:30:03.419626    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801\": container with ID starting with 5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801 not found: ID does not exist" containerID="5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801"
Sep 17 23:30:03.419789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.419666    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801"} err="failed to get container status \"5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801\": rpc error: code = NotFound desc = could not find container \"5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801\": container with ID starting with 5a981c787e8337d0d9506962326bdbaf1047379b8a5726e7322f812ecc387801 not found: ID does not exist"
Sep 17 23:30:03.430641 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.430609    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"]
Sep 17 23:30:03.437090 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.437058    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-0b4d93-84f9cdfdb6-p7nsd"]
Sep 17 23:30:03.524282 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.524258    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk"
Sep 17 23:30:03.579750 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.579727    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"
Sep 17 23:30:03.607838 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.607604    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"
Sep 17 23:30:03.611248 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.611227    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"
Sep 17 23:30:03.994756 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:03.994722    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b4b9fa2-6693-4be0-bc8b-c639d5f3191d" path="/var/lib/kubelet/pods/9b4b9fa2-6693-4be0-bc8b-c639d5f3191d/volumes"
Sep 17 23:30:04.376687 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.376649    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq" event={"ID":"f379f212-b36f-4b2f-9595-e6d5163f1dbd","Type":"ContainerDied","Data":"5a312286118cbc67609693bda08e3b895f5c28fd8f3a9431a6f64d3161ea729b"}
Sep 17 23:30:04.377067 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.376667    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"
Sep 17 23:30:04.377067 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.376731    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a312286118cbc67609693bda08e3b895f5c28fd8f3a9431a6f64d3161ea729b"
Sep 17 23:30:04.378071 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.378047    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"
Sep 17 23:30:04.378198 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.378073    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn" event={"ID":"518a9c6d-776e-4289-8a0a-dca6b1ad17db","Type":"ContainerDied","Data":"3fe2a7049e14e73feaf85edfdbe914ac7294f0d5fadb743ac52d3c01ce6d92cf"}
Sep 17 23:30:04.378198 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.378138    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fe2a7049e14e73feaf85edfdbe914ac7294f0d5fadb743ac52d3c01ce6d92cf"
Sep 17 23:30:04.379662 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.379630    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk"
Sep 17 23:30:04.379769 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.379630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828130-zgdhk" event={"ID":"2736b23a-1440-4d3f-b591-a519b8842554","Type":"ContainerDied","Data":"52565b61eaad1778135931b0c317a4846016b44e7e9483d7239c5a58a3261020"}
Sep 17 23:30:04.379769 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.379755    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52565b61eaad1778135931b0c317a4846016b44e7e9483d7239c5a58a3261020"
Sep 17 23:30:04.381965 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.381942    2581 generic.go:356] "Generic (PLEG): container finished" podID="7090575f-4f2e-493f-8c13-45005510eb90" containerID="d71c26cb2fee89121c3da1d9cf3d8db41c08da410c22d4a15095871d4ec6813d" exitCode=0
Sep 17 23:30:04.382079 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.382055    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq" event={"ID":"7090575f-4f2e-493f-8c13-45005510eb90","Type":"ContainerDied","Data":"d71c26cb2fee89121c3da1d9cf3d8db41c08da410c22d4a15095871d4ec6813d"}
Sep 17 23:30:04.382166 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.382103    2581 scope.go:117] "RemoveContainer" containerID="411075d30ba93cb52ca96af617e3c70d43326c7bef3f2ceb5f55250fc4d01a11"
Sep 17 23:30:04.385267 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.385246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7" event={"ID":"25cf16e7-a51b-4e40-aac3-6060e94cc265","Type":"ContainerDied","Data":"fe92db77cd67562d42f65e345628199fa143b83262240dffc707e7dbcaaca3bb"}
Sep 17 23:30:04.385370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.385285    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe92db77cd67562d42f65e345628199fa143b83262240dffc707e7dbcaaca3bb"
Sep 17 23:30:04.385370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.385316    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"
Sep 17 23:30:04.525951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:04.525931    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"
Sep 17 23:30:05.393489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:05.393460    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"
Sep 17 23:30:05.393489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:05.393486    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq" event={"ID":"7090575f-4f2e-493f-8c13-45005510eb90","Type":"ContainerDied","Data":"285effa30fbbf00788a50e07a0f1b8a9ce4838bf87ba3417252d55785e2e9d86"}
Sep 17 23:30:05.393937 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:05.393524    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="285effa30fbbf00788a50e07a0f1b8a9ce4838bf87ba3417252d55785e2e9d86"
Sep 17 23:30:06.415450 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.415406    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt" podUID="09493d74-f4ba-4817-b17c-36a7134a5dfc" containerName="main" containerID="cri-o://e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d" gracePeriod=45
Sep 17 23:30:06.426346 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.426297    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf" podUID="d4ec9362-ae8d-429d-9275-06fa40a9263d" containerName="main" containerID="cri-o://680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2" gracePeriod=45
Sep 17 23:30:06.676165 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.676062    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:30:06.705218 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.705193    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:30:06.790071 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790041    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-tmp-dir\") pod \"09493d74-f4ba-4817-b17c-36a7134a5dfc\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") "
Sep 17 23:30:06.790253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790094    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l9mw\" (UniqueName: \"kubernetes.io/projected/09493d74-f4ba-4817-b17c-36a7134a5dfc-kube-api-access-2l9mw\") pod \"09493d74-f4ba-4817-b17c-36a7134a5dfc\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") "
Sep 17 23:30:06.790253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790118    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09493d74-f4ba-4817-b17c-36a7134a5dfc-tls-certs\") pod \"09493d74-f4ba-4817-b17c-36a7134a5dfc\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") "
Sep 17 23:30:06.790253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790202    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-home\") pod \"d4ec9362-ae8d-429d-9275-06fa40a9263d\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") "
Sep 17 23:30:06.790253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790234    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-home\") pod \"09493d74-f4ba-4817-b17c-36a7134a5dfc\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") "
Sep 17 23:30:06.790489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790260    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ec9362-ae8d-429d-9275-06fa40a9263d-tls-certs\") pod \"d4ec9362-ae8d-429d-9275-06fa40a9263d\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") "
Sep 17 23:30:06.790489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790295    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-model-cache\") pod \"09493d74-f4ba-4817-b17c-36a7134a5dfc\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") "
Sep 17 23:30:06.790489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790335    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-tmp-dir\") pod \"d4ec9362-ae8d-429d-9275-06fa40a9263d\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") "
Sep 17 23:30:06.790489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790389    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg5lr\" (UniqueName: \"kubernetes.io/projected/d4ec9362-ae8d-429d-9275-06fa40a9263d-kube-api-access-pg5lr\") pod \"d4ec9362-ae8d-429d-9275-06fa40a9263d\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") "
Sep 17 23:30:06.790489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790435    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-model-cache\") pod \"d4ec9362-ae8d-429d-9275-06fa40a9263d\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") "
Sep 17 23:30:06.790489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790473    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-dshm\") pod \"d4ec9362-ae8d-429d-9275-06fa40a9263d\" (UID: \"d4ec9362-ae8d-429d-9275-06fa40a9263d\") "
Sep 17 23:30:06.790489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790472    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09493d74-f4ba-4817-b17c-36a7134a5dfc" (UID: "09493d74-f4ba-4817-b17c-36a7134a5dfc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:06.790820 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790513    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-dshm\") pod \"09493d74-f4ba-4817-b17c-36a7134a5dfc\" (UID: \"09493d74-f4ba-4817-b17c-36a7134a5dfc\") "
Sep 17 23:30:06.790820 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790649    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d4ec9362-ae8d-429d-9275-06fa40a9263d" (UID: "d4ec9362-ae8d-429d-9275-06fa40a9263d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:06.790962 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790935    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.791028 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790969    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.791028 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.790969    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-model-cache" (OuterVolumeSpecName: "model-cache") pod "09493d74-f4ba-4817-b17c-36a7134a5dfc" (UID: "09493d74-f4ba-4817-b17c-36a7134a5dfc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:06.791219 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.791196    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-home" (OuterVolumeSpecName: "home") pod "d4ec9362-ae8d-429d-9275-06fa40a9263d" (UID: "d4ec9362-ae8d-429d-9275-06fa40a9263d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:06.791282 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.791249    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d4ec9362-ae8d-429d-9275-06fa40a9263d" (UID: "d4ec9362-ae8d-429d-9275-06fa40a9263d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:06.791375 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.791355    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-home" (OuterVolumeSpecName: "home") pod "09493d74-f4ba-4817-b17c-36a7134a5dfc" (UID: "09493d74-f4ba-4817-b17c-36a7134a5dfc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:06.793417 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.793306    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09493d74-f4ba-4817-b17c-36a7134a5dfc-kube-api-access-2l9mw" (OuterVolumeSpecName: "kube-api-access-2l9mw") pod "09493d74-f4ba-4817-b17c-36a7134a5dfc" (UID: "09493d74-f4ba-4817-b17c-36a7134a5dfc"). InnerVolumeSpecName "kube-api-access-2l9mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:30:06.793417 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.793373    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-dshm" (OuterVolumeSpecName: "dshm") pod "09493d74-f4ba-4817-b17c-36a7134a5dfc" (UID: "09493d74-f4ba-4817-b17c-36a7134a5dfc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:06.793688 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.793658    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4ec9362-ae8d-429d-9275-06fa40a9263d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d4ec9362-ae8d-429d-9275-06fa40a9263d" (UID: "d4ec9362-ae8d-429d-9275-06fa40a9263d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:30:06.793688 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.793661    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ec9362-ae8d-429d-9275-06fa40a9263d-kube-api-access-pg5lr" (OuterVolumeSpecName: "kube-api-access-pg5lr") pod "d4ec9362-ae8d-429d-9275-06fa40a9263d" (UID: "d4ec9362-ae8d-429d-9275-06fa40a9263d"). InnerVolumeSpecName "kube-api-access-pg5lr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:30:06.793850 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.793718    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09493d74-f4ba-4817-b17c-36a7134a5dfc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09493d74-f4ba-4817-b17c-36a7134a5dfc" (UID: "09493d74-f4ba-4817-b17c-36a7134a5dfc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:30:06.793850 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.793731    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-dshm" (OuterVolumeSpecName: "dshm") pod "d4ec9362-ae8d-429d-9275-06fa40a9263d" (UID: "d4ec9362-ae8d-429d-9275-06fa40a9263d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:30:06.892365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892331    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg5lr\" (UniqueName: \"kubernetes.io/projected/d4ec9362-ae8d-429d-9275-06fa40a9263d-kube-api-access-pg5lr\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892358    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892365 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892368    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892377    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892386    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l9mw\" (UniqueName: \"kubernetes.io/projected/09493d74-f4ba-4817-b17c-36a7134a5dfc-kube-api-access-2l9mw\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892394    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09493d74-f4ba-4817-b17c-36a7134a5dfc-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892402    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4ec9362-ae8d-429d-9275-06fa40a9263d-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892409    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892417    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4ec9362-ae8d-429d-9275-06fa40a9263d-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:06.892598 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:06.892424    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09493d74-f4ba-4817-b17c-36a7134a5dfc-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:07.407691 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.407655    2581 generic.go:356] "Generic (PLEG): container finished" podID="09493d74-f4ba-4817-b17c-36a7134a5dfc" containerID="e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d" exitCode=0
Sep 17 23:30:07.407880 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.407701    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"
Sep 17 23:30:07.407880 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.407751    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt" event={"ID":"09493d74-f4ba-4817-b17c-36a7134a5dfc","Type":"ContainerDied","Data":"e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d"}
Sep 17 23:30:07.407880 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.407802    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt" event={"ID":"09493d74-f4ba-4817-b17c-36a7134a5dfc","Type":"ContainerDied","Data":"3dd0b94db264376a3cd2f8be3f4ab34e92e3290565a26e1beb19e380d87ecc26"}
Sep 17 23:30:07.407880 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.407824    2581 scope.go:117] "RemoveContainer" containerID="e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d"
Sep 17 23:30:07.409364 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.409344    2581 generic.go:356] "Generic (PLEG): container finished" podID="d4ec9362-ae8d-429d-9275-06fa40a9263d" containerID="680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2" exitCode=0
Sep 17 23:30:07.409449 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.409380    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"
Sep 17 23:30:07.409449 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.409405    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf" event={"ID":"d4ec9362-ae8d-429d-9275-06fa40a9263d","Type":"ContainerDied","Data":"680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2"}
Sep 17 23:30:07.409449 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.409446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf" event={"ID":"d4ec9362-ae8d-429d-9275-06fa40a9263d","Type":"ContainerDied","Data":"f465a42a87b90885160d6749546bc450449a61c1824edc83e445d726a0a25174"}
Sep 17 23:30:07.420116 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.419917    2581 scope.go:117] "RemoveContainer" containerID="e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d"
Sep 17 23:30:07.420393 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:30:07.420280    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d\": container with ID starting with e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d not found: ID does not exist" containerID="e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d"
Sep 17 23:30:07.420393 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.420311    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d"} err="failed to get container status \"e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d\": rpc error: code = NotFound desc = could not find container \"e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d\": container with ID starting with e1593e47182f6e3a17732dd0a48bd7523ebc9afd2d153b71cfde624cf4d74b9d not found: ID does not exist"
Sep 17 23:30:07.420393 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.420330    2581 scope.go:117] "RemoveContainer" containerID="680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2"
Sep 17 23:30:07.429467 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.429448    2581 scope.go:117] "RemoveContainer" containerID="680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2"
Sep 17 23:30:07.429793 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:30:07.429774    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2\": container with ID starting with 680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2 not found: ID does not exist" containerID="680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2"
Sep 17 23:30:07.429847 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.429800    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2"} err="failed to get container status \"680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2\": rpc error: code = NotFound desc = could not find container \"680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2\": container with ID starting with 680feab954fb066579b9dca218c8537b5d87297cd5118c56bb7de01dcf5223a2 not found: ID does not exist"
Sep 17 23:30:07.438046 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.438018    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"]
Sep 17 23:30:07.441950 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.441929    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/distinct-2-w0-0b4d93-kserve-6647f5c686-2klwt"]
Sep 17 23:30:07.452397 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.452370    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"]
Sep 17 23:30:07.456262 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.456240    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-0b4d93/embedding-w0-0b4d93-kserve-7b98bbd9d-rh6jf"]
Sep 17 23:30:07.994894 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.994860    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09493d74-f4ba-4817-b17c-36a7134a5dfc" path="/var/lib/kubelet/pods/09493d74-f4ba-4817-b17c-36a7134a5dfc/volumes"
Sep 17 23:30:07.995299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:07.995285    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4ec9362-ae8d-429d-9275-06fa40a9263d" path="/var/lib/kubelet/pods/d4ec9362-ae8d-429d-9275-06fa40a9263d/volumes"
Sep 17 23:30:13.999992 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:13.999961    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"]
Sep 17 23:30:14.000470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000452    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25cf16e7-a51b-4e40-aac3-6060e94cc265" containerName="cleanup"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000473    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="25cf16e7-a51b-4e40-aac3-6060e94cc265" containerName="cleanup"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000488    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2736b23a-1440-4d3f-b591-a519b8842554" containerName="cleanup"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000494    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2736b23a-1440-4d3f-b591-a519b8842554" containerName="cleanup"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000510    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7090575f-4f2e-493f-8c13-45005510eb90" containerName="cleanup"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000515    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7090575f-4f2e-493f-8c13-45005510eb90" containerName="cleanup"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000524    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09493d74-f4ba-4817-b17c-36a7134a5dfc" containerName="main"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000530    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="09493d74-f4ba-4817-b17c-36a7134a5dfc" containerName="main"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000537    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f379f212-b36f-4b2f-9595-e6d5163f1dbd" containerName="cleanup"
Sep 17 23:30:14.000537 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000542    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f379f212-b36f-4b2f-9595-e6d5163f1dbd" containerName="cleanup"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000564    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b4b9fa2-6693-4be0-bc8b-c639d5f3191d" containerName="payload-pre-processing"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000570    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b4b9fa2-6693-4be0-bc8b-c639d5f3191d" containerName="payload-pre-processing"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000578    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ec9362-ae8d-429d-9275-06fa40a9263d" containerName="main"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000583    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ec9362-ae8d-429d-9275-06fa40a9263d" containerName="main"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000591    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="518a9c6d-776e-4289-8a0a-dca6b1ad17db" containerName="cleanup"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000596    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="518a9c6d-776e-4289-8a0a-dca6b1ad17db" containerName="cleanup"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000684    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7090575f-4f2e-493f-8c13-45005510eb90" containerName="cleanup"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000692    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="25cf16e7-a51b-4e40-aac3-6060e94cc265" containerName="cleanup"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000699    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ec9362-ae8d-429d-9275-06fa40a9263d" containerName="main"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000704    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="518a9c6d-776e-4289-8a0a-dca6b1ad17db" containerName="cleanup"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000710    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b4b9fa2-6693-4be0-bc8b-c639d5f3191d" containerName="payload-pre-processing"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000717    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f379f212-b36f-4b2f-9595-e6d5163f1dbd" containerName="cleanup"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000722    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2736b23a-1440-4d3f-b591-a519b8842554" containerName="cleanup"
Sep 17 23:30:14.000827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.000727    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="09493d74-f4ba-4817-b17c-36a7134a5dfc" containerName="main"
Sep 17 23:30:14.003812 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.003797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"
Sep 17 23:30:14.006220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.006197    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:30:14.006323 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.006236    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:30:14.011532 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.011506    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"]
Sep 17 23:30:14.071987 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.071952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c78126b-fa5d-4e57-9ce5-61dfc75ea152-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-eb0661-whfzf\" (UID: \"2c78126b-fa5d-4e57-9ce5-61dfc75ea152\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"
Sep 17 23:30:14.173088 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.173044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c78126b-fa5d-4e57-9ce5-61dfc75ea152-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-eb0661-whfzf\" (UID: \"2c78126b-fa5d-4e57-9ce5-61dfc75ea152\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"
Sep 17 23:30:14.173683 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.173665    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c78126b-fa5d-4e57-9ce5-61dfc75ea152-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-eb0661-whfzf\" (UID: \"2c78126b-fa5d-4e57-9ce5-61dfc75ea152\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"
Sep 17 23:30:14.315696 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.315606    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"
Sep 17 23:30:14.600448 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:14.600426    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"]
Sep 17 23:30:14.601159 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:30:14.601107    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c78126b_fa5d_4e57_9ce5_61dfc75ea152.slice/crio-628929f58de0c72e979f5b7c014e202199a5f9b7d2b7dd0d67313ce2958e25d9 WatchSource:0}: Error finding container 628929f58de0c72e979f5b7c014e202199a5f9b7d2b7dd0d67313ce2958e25d9: Status 404 returned error can't find the container with id 628929f58de0c72e979f5b7c014e202199a5f9b7d2b7dd0d67313ce2958e25d9
Sep 17 23:30:15.481977 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:15.481943    2581 generic.go:356] "Generic (PLEG): container finished" podID="2c78126b-fa5d-4e57-9ce5-61dfc75ea152" containerID="bf023fc17d2d146b1e257e1ca2c5c827988d317d593aeee57f30abc28afadf1c" exitCode=0
Sep 17 23:30:15.482408 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:15.482028    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf" event={"ID":"2c78126b-fa5d-4e57-9ce5-61dfc75ea152","Type":"ContainerDied","Data":"bf023fc17d2d146b1e257e1ca2c5c827988d317d593aeee57f30abc28afadf1c"}
Sep 17 23:30:15.482408 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:15.482092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf" event={"ID":"2c78126b-fa5d-4e57-9ce5-61dfc75ea152","Type":"ContainerStarted","Data":"628929f58de0c72e979f5b7c014e202199a5f9b7d2b7dd0d67313ce2958e25d9"}
Sep 17 23:30:16.635049 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:16.635023    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"
Sep 17 23:30:16.703144 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:16.703092    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c78126b-fa5d-4e57-9ce5-61dfc75ea152-maas-api-service-ca\") pod \"2c78126b-fa5d-4e57-9ce5-61dfc75ea152\" (UID: \"2c78126b-fa5d-4e57-9ce5-61dfc75ea152\") "
Sep 17 23:30:16.703441 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:16.703419    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c78126b-fa5d-4e57-9ce5-61dfc75ea152-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c78126b-fa5d-4e57-9ce5-61dfc75ea152" (UID: "2c78126b-fa5d-4e57-9ce5-61dfc75ea152"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:30:16.804353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:16.804273    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c78126b-fa5d-4e57-9ce5-61dfc75ea152-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:17.496712 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:17.496683    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"
Sep 17 23:30:17.496881 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:17.496686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf" event={"ID":"2c78126b-fa5d-4e57-9ce5-61dfc75ea152","Type":"ContainerDied","Data":"628929f58de0c72e979f5b7c014e202199a5f9b7d2b7dd0d67313ce2958e25d9"}
Sep 17 23:30:17.496881 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:17.496826    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="628929f58de0c72e979f5b7c014e202199a5f9b7d2b7dd0d67313ce2958e25d9"
Sep 17 23:30:22.350050 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:22.350015    2581 scope.go:117] "RemoveContainer" containerID="97a96317f010eaaed65875e8da0e31eeb1583bf3b8cdbd0c7deaf17d86098e97"
Sep 17 23:30:29.717716 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:29.717676    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"]
Sep 17 23:30:29.720552 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:29.720529    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-eb0661-29828130-mpplq"]
Sep 17 23:30:29.742766 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:29.742735    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"]
Sep 17 23:30:29.743232 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:29.743182    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb" podUID="73ae9e6f-454c-428a-b2cb-7d65f15d18fd" containerName="payload-processing" containerID="cri-o://66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90" gracePeriod=30
Sep 17 23:30:29.754069 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:29.754036    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"]
Sep 17 23:30:29.754344 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:29.754294    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9" podUID="bb663080-1e12-45c1-af0a-02671cafc675" containerName="payload-pre-processing" containerID="cri-o://fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4" gracePeriod=30
Sep 17 23:30:29.925880 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:29.925844    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"]
Sep 17 23:30:29.940434 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:29.940404    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-eb0661-whfzf"]
Sep 17 23:30:30.012339 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.011467    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c78126b-fa5d-4e57-9ce5-61dfc75ea152" path="/var/lib/kubelet/pods/2c78126b-fa5d-4e57-9ce5-61dfc75ea152/volumes"
Sep 17 23:30:30.012339 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.011953    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f379f212-b36f-4b2f-9595-e6d5163f1dbd" path="/var/lib/kubelet/pods/f379f212-b36f-4b2f-9595-e6d5163f1dbd/volumes"
Sep 17 23:30:30.027596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.027562    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:30:30.073432 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.073408    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:30:30.140241 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.140211    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb663080-1e12-45c1-af0a-02671cafc675-plugins-config-volume\") pod \"bb663080-1e12-45c1-af0a-02671cafc675\" (UID: \"bb663080-1e12-45c1-af0a-02671cafc675\") "
Sep 17 23:30:30.140241 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.140245    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-plugins-config-volume\") pod \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\" (UID: \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\") "
Sep 17 23:30:30.140486 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.140362    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sp95\" (UniqueName: \"kubernetes.io/projected/bb663080-1e12-45c1-af0a-02671cafc675-kube-api-access-5sp95\") pod \"bb663080-1e12-45c1-af0a-02671cafc675\" (UID: \"bb663080-1e12-45c1-af0a-02671cafc675\") "
Sep 17 23:30:30.140486 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.140437    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjb4d\" (UniqueName: \"kubernetes.io/projected/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-kube-api-access-rjb4d\") pod \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\" (UID: \"73ae9e6f-454c-428a-b2cb-7d65f15d18fd\") "
Sep 17 23:30:30.140679 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.140656    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb663080-1e12-45c1-af0a-02671cafc675-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb663080-1e12-45c1-af0a-02671cafc675" (UID: "bb663080-1e12-45c1-af0a-02671cafc675"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:30:30.140758 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.140697    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "73ae9e6f-454c-428a-b2cb-7d65f15d18fd" (UID: "73ae9e6f-454c-428a-b2cb-7d65f15d18fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:30:30.140898 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.140880    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb663080-1e12-45c1-af0a-02671cafc675-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:30.140946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.140903    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:30.142793 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.142762    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-kube-api-access-rjb4d" (OuterVolumeSpecName: "kube-api-access-rjb4d") pod "73ae9e6f-454c-428a-b2cb-7d65f15d18fd" (UID: "73ae9e6f-454c-428a-b2cb-7d65f15d18fd"). InnerVolumeSpecName "kube-api-access-rjb4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:30:30.142942 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.142922    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb663080-1e12-45c1-af0a-02671cafc675-kube-api-access-5sp95" (OuterVolumeSpecName: "kube-api-access-5sp95") pod "bb663080-1e12-45c1-af0a-02671cafc675" (UID: "bb663080-1e12-45c1-af0a-02671cafc675"). InnerVolumeSpecName "kube-api-access-5sp95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:30:30.241588 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.241511    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sp95\" (UniqueName: \"kubernetes.io/projected/bb663080-1e12-45c1-af0a-02671cafc675-kube-api-access-5sp95\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:30.241588 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.241542    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjb4d\" (UniqueName: \"kubernetes.io/projected/73ae9e6f-454c-428a-b2cb-7d65f15d18fd-kube-api-access-rjb4d\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:30.579993 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.579900    2581 generic.go:356] "Generic (PLEG): container finished" podID="73ae9e6f-454c-428a-b2cb-7d65f15d18fd" containerID="66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90" exitCode=0
Sep 17 23:30:30.579993 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.579943    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"
Sep 17 23:30:30.580253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.579986    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb" event={"ID":"73ae9e6f-454c-428a-b2cb-7d65f15d18fd","Type":"ContainerDied","Data":"66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90"}
Sep 17 23:30:30.580253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.580045    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb" event={"ID":"73ae9e6f-454c-428a-b2cb-7d65f15d18fd","Type":"ContainerDied","Data":"e1367d6e254da5942b95dccb380942b7fb656108f3cfedf4f5561e2ef1e596a0"}
Sep 17 23:30:30.580253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.580077    2581 scope.go:117] "RemoveContainer" containerID="66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90"
Sep 17 23:30:30.581533 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.581508    2581 generic.go:356] "Generic (PLEG): container finished" podID="bb663080-1e12-45c1-af0a-02671cafc675" containerID="fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4" exitCode=0
Sep 17 23:30:30.581666 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.581551    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"
Sep 17 23:30:30.581666 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.581575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9" event={"ID":"bb663080-1e12-45c1-af0a-02671cafc675","Type":"ContainerDied","Data":"fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4"}
Sep 17 23:30:30.581666 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.581622    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9" event={"ID":"bb663080-1e12-45c1-af0a-02671cafc675","Type":"ContainerDied","Data":"708984d4dbd4e54cc244149cd82e81e64005c72e1aa689ca42ae7a75d9cdc7ea"}
Sep 17 23:30:30.596652 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.596625    2581 scope.go:117] "RemoveContainer" containerID="66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90"
Sep 17 23:30:30.597038 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:30:30.597006    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90\": container with ID starting with 66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90 not found: ID does not exist" containerID="66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90"
Sep 17 23:30:30.597173 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.597090    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90"} err="failed to get container status \"66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90\": rpc error: code = NotFound desc = could not find container \"66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90\": container with ID starting with 66eb4bafc92785c9a0386e2007a4eacff84c18c919b8fc726615de0b4dfd5a90 not found: ID does not exist"
Sep 17 23:30:30.597173 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.597157    2581 scope.go:117] "RemoveContainer" containerID="fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4"
Sep 17 23:30:30.607046 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.607022    2581 scope.go:117] "RemoveContainer" containerID="fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4"
Sep 17 23:30:30.607353 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:30:30.607334    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4\": container with ID starting with fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4 not found: ID does not exist" containerID="fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4"
Sep 17 23:30:30.607426 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.607367    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4"} err="failed to get container status \"fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4\": rpc error: code = NotFound desc = could not find container \"fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4\": container with ID starting with fe7f5db5ca0be8b41cdd697497eb5397ce6788cc67e3338ce69930a25f618ce4 not found: ID does not exist"
Sep 17 23:30:30.616470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.616435    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"]
Sep 17 23:30:30.623969 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.623933    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eb0661-56654bb8c5-w6dmb"]
Sep 17 23:30:30.637942 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.637918    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"]
Sep 17 23:30:30.642646 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:30.642623    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eb0661-9557f694b-lnwp9"]
Sep 17 23:30:31.994216 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:31.994179    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73ae9e6f-454c-428a-b2cb-7d65f15d18fd" path="/var/lib/kubelet/pods/73ae9e6f-454c-428a-b2cb-7d65f15d18fd/volumes"
Sep 17 23:30:31.994616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:31.994601    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb663080-1e12-45c1-af0a-02671cafc675" path="/var/lib/kubelet/pods/bb663080-1e12-45c1-af0a-02671cafc675/volumes"
Sep 17 23:30:37.015908 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.015872    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"]
Sep 17 23:30:37.016332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016296    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb663080-1e12-45c1-af0a-02671cafc675" containerName="payload-pre-processing"
Sep 17 23:30:37.016332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016310    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb663080-1e12-45c1-af0a-02671cafc675" containerName="payload-pre-processing"
Sep 17 23:30:37.016332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016319    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ae9e6f-454c-428a-b2cb-7d65f15d18fd" containerName="payload-processing"
Sep 17 23:30:37.016332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016324    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ae9e6f-454c-428a-b2cb-7d65f15d18fd" containerName="payload-processing"
Sep 17 23:30:37.016332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016332    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7090575f-4f2e-493f-8c13-45005510eb90" containerName="cleanup"
Sep 17 23:30:37.016495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016338    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7090575f-4f2e-493f-8c13-45005510eb90" containerName="cleanup"
Sep 17 23:30:37.016495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016360    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c78126b-fa5d-4e57-9ce5-61dfc75ea152" containerName="revoke-keys"
Sep 17 23:30:37.016495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016365    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c78126b-fa5d-4e57-9ce5-61dfc75ea152" containerName="revoke-keys"
Sep 17 23:30:37.016495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016424    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c78126b-fa5d-4e57-9ce5-61dfc75ea152" containerName="revoke-keys"
Sep 17 23:30:37.016495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016432    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb663080-1e12-45c1-af0a-02671cafc675" containerName="payload-pre-processing"
Sep 17 23:30:37.016495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016439    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7090575f-4f2e-493f-8c13-45005510eb90" containerName="cleanup"
Sep 17 23:30:37.016495 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.016447    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="73ae9e6f-454c-428a-b2cb-7d65f15d18fd" containerName="payload-processing"
Sep 17 23:30:37.021096 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.021078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"
Sep 17 23:30:37.023998 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.023976    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:30:37.025199 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.025173    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:30:37.029380 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.029359    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"]
Sep 17 23:30:37.104585 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.104546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5b099c-520d-45cd-81fc-35e8a74f23f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg\" (UID: \"be5b099c-520d-45cd-81fc-35e8a74f23f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"
Sep 17 23:30:37.206045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.206002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5b099c-520d-45cd-81fc-35e8a74f23f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg\" (UID: \"be5b099c-520d-45cd-81fc-35e8a74f23f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"
Sep 17 23:30:37.206658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.206637    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5b099c-520d-45cd-81fc-35e8a74f23f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg\" (UID: \"be5b099c-520d-45cd-81fc-35e8a74f23f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"
Sep 17 23:30:37.333521 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.333441    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"
Sep 17 23:30:37.610948 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:30:37.610917    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe5b099c_520d_45cd_81fc_35e8a74f23f6.slice/crio-a7d49e2c4feeb3dc410939a9c86614b91db56def2d0493a7765c3f5f688fdc4a WatchSource:0}: Error finding container a7d49e2c4feeb3dc410939a9c86614b91db56def2d0493a7765c3f5f688fdc4a: Status 404 returned error can't find the container with id a7d49e2c4feeb3dc410939a9c86614b91db56def2d0493a7765c3f5f688fdc4a
Sep 17 23:30:37.611447 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.611417    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"]
Sep 17 23:30:37.633703 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:37.633673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg" event={"ID":"be5b099c-520d-45cd-81fc-35e8a74f23f6","Type":"ContainerStarted","Data":"a7d49e2c4feeb3dc410939a9c86614b91db56def2d0493a7765c3f5f688fdc4a"}
Sep 17 23:30:38.642233 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:38.642200    2581 generic.go:356] "Generic (PLEG): container finished" podID="be5b099c-520d-45cd-81fc-35e8a74f23f6" containerID="0e95b7928bdff5f88fa7905642e69cd327f03fdfa1d05ba523a418f3ff60117e" exitCode=0
Sep 17 23:30:38.642640 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:38.642290    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg" event={"ID":"be5b099c-520d-45cd-81fc-35e8a74f23f6","Type":"ContainerDied","Data":"0e95b7928bdff5f88fa7905642e69cd327f03fdfa1d05ba523a418f3ff60117e"}
Sep 17 23:30:39.788241 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:39.788217    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"
Sep 17 23:30:39.832730 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:39.832694    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5b099c-520d-45cd-81fc-35e8a74f23f6-maas-api-service-ca\") pod \"be5b099c-520d-45cd-81fc-35e8a74f23f6\" (UID: \"be5b099c-520d-45cd-81fc-35e8a74f23f6\") "
Sep 17 23:30:39.833043 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:39.833020    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be5b099c-520d-45cd-81fc-35e8a74f23f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "be5b099c-520d-45cd-81fc-35e8a74f23f6" (UID: "be5b099c-520d-45cd-81fc-35e8a74f23f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:30:39.934568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:39.934479    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5b099c-520d-45cd-81fc-35e8a74f23f6-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:30:40.656304 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:40.656260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg" event={"ID":"be5b099c-520d-45cd-81fc-35e8a74f23f6","Type":"ContainerDied","Data":"a7d49e2c4feeb3dc410939a9c86614b91db56def2d0493a7765c3f5f688fdc4a"}
Sep 17 23:30:40.656513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:40.656365    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7d49e2c4feeb3dc410939a9c86614b91db56def2d0493a7765c3f5f688fdc4a"
Sep 17 23:30:40.656513 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:40.656437    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"
Sep 17 23:30:52.628663 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:52.628622    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"]
Sep 17 23:30:52.632526 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:52.632501    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-00d34b-29828130-bvbtq"]
Sep 17 23:30:52.827194 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:52.827151    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"]
Sep 17 23:30:52.831457 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:52.831427    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-00d34b-9s4qg"]
Sep 17 23:30:53.993369 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:53.993328    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7090575f-4f2e-493f-8c13-45005510eb90" path="/var/lib/kubelet/pods/7090575f-4f2e-493f-8c13-45005510eb90/volumes"
Sep 17 23:30:53.993840 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:53.993816    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be5b099c-520d-45cd-81fc-35e8a74f23f6" path="/var/lib/kubelet/pods/be5b099c-520d-45cd-81fc-35e8a74f23f6/volumes"
Sep 17 23:30:59.622351 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.622318    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"]
Sep 17 23:30:59.622816 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.622749    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be5b099c-520d-45cd-81fc-35e8a74f23f6" containerName="revoke-keys"
Sep 17 23:30:59.622816 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.622760    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="be5b099c-520d-45cd-81fc-35e8a74f23f6" containerName="revoke-keys"
Sep 17 23:30:59.622892 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.622821    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="be5b099c-520d-45cd-81fc-35e8a74f23f6" containerName="revoke-keys"
Sep 17 23:30:59.626330 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.626312    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"
Sep 17 23:30:59.628945 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.628925    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:30:59.629043 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.628977    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:30:59.634977 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.634954    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"]
Sep 17 23:30:59.733373 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.733329    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b944ac16-da00-45b3-9bb0-08255e071de9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf\" (UID: \"b944ac16-da00-45b3-9bb0-08255e071de9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"
Sep 17 23:30:59.834143 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.834095    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b944ac16-da00-45b3-9bb0-08255e071de9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf\" (UID: \"b944ac16-da00-45b3-9bb0-08255e071de9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"
Sep 17 23:30:59.834719 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.834698    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b944ac16-da00-45b3-9bb0-08255e071de9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf\" (UID: \"b944ac16-da00-45b3-9bb0-08255e071de9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"
Sep 17 23:30:59.940518 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:30:59.940425    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"
Sep 17 23:31:00.219460 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:00.219413    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"]
Sep 17 23:31:00.221309 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:31:00.221283    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb944ac16_da00_45b3_9bb0_08255e071de9.slice/crio-abf0bd15d231d385bbedcedf298c2a96535f4327d38a69e8adfe7c0df6483c56 WatchSource:0}: Error finding container abf0bd15d231d385bbedcedf298c2a96535f4327d38a69e8adfe7c0df6483c56: Status 404 returned error can't find the container with id abf0bd15d231d385bbedcedf298c2a96535f4327d38a69e8adfe7c0df6483c56
Sep 17 23:31:00.791159 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:00.791104    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf" event={"ID":"b944ac16-da00-45b3-9bb0-08255e071de9","Type":"ContainerStarted","Data":"485582dd520918c3e6943587412d16cf93f4e9fbfab7a328dc15d9bf5983a192"}
Sep 17 23:31:00.791530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:00.791177    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf" event={"ID":"b944ac16-da00-45b3-9bb0-08255e071de9","Type":"ContainerStarted","Data":"abf0bd15d231d385bbedcedf298c2a96535f4327d38a69e8adfe7c0df6483c56"}
Sep 17 23:31:00.807741 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:00.807693    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf" podStartSLOduration=1.8076786660000002 podStartE2EDuration="1.807678666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:30:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:31:00.807314099 +0000 UTC m=+1480.811505939" watchObservedRunningTime="2026-09-17 23:31:00.807678666 +0000 UTC m=+1480.811870505"
Sep 17 23:31:01.798866 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:01.798830    2581 generic.go:356] "Generic (PLEG): container finished" podID="b944ac16-da00-45b3-9bb0-08255e071de9" containerID="485582dd520918c3e6943587412d16cf93f4e9fbfab7a328dc15d9bf5983a192" exitCode=0
Sep 17 23:31:01.799353 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:01.798916    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf" event={"ID":"b944ac16-da00-45b3-9bb0-08255e071de9","Type":"ContainerDied","Data":"485582dd520918c3e6943587412d16cf93f4e9fbfab7a328dc15d9bf5983a192"}
Sep 17 23:31:02.947393 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:02.947371    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"
Sep 17 23:31:03.065230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:03.065111    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b944ac16-da00-45b3-9bb0-08255e071de9-maas-api-service-ca\") pod \"b944ac16-da00-45b3-9bb0-08255e071de9\" (UID: \"b944ac16-da00-45b3-9bb0-08255e071de9\") "
Sep 17 23:31:03.065520 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:03.065497    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b944ac16-da00-45b3-9bb0-08255e071de9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b944ac16-da00-45b3-9bb0-08255e071de9" (UID: "b944ac16-da00-45b3-9bb0-08255e071de9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:31:03.166640 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:03.166603    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b944ac16-da00-45b3-9bb0-08255e071de9-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:03.812924 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:03.812893    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"
Sep 17 23:31:03.813115 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:03.812930    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf" event={"ID":"b944ac16-da00-45b3-9bb0-08255e071de9","Type":"ContainerDied","Data":"abf0bd15d231d385bbedcedf298c2a96535f4327d38a69e8adfe7c0df6483c56"}
Sep 17 23:31:03.813115 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:03.812976    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abf0bd15d231d385bbedcedf298c2a96535f4327d38a69e8adfe7c0df6483c56"
Sep 17 23:31:05.095563 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:05.095524    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"]
Sep 17 23:31:10.069822 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.069786    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"]
Sep 17 23:31:10.072736 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.072706    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-09c1c1-29828130-9tmm7"]
Sep 17 23:31:10.096257 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.096223    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"]
Sep 17 23:31:10.096742 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.096678    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv" podUID="d229854a-a04b-42fc-ace8-a5ac99979da8" containerName="payload-processing" containerID="cri-o://bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b" gracePeriod=30
Sep 17 23:31:10.100936 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.100785    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"]
Sep 17 23:31:10.101160 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.101066    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg" podUID="a4b06b80-0e03-41f5-b00b-a131ae8e98cb" containerName="payload-pre-processing" containerID="cri-o://510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603" gracePeriod=30
Sep 17 23:31:10.255277 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.255222    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"]
Sep 17 23:31:10.259611 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.259583    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-09c1c1-nglvf"]
Sep 17 23:31:10.388841 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.388816    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:31:10.391855 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.391835    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:31:10.540386 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.540351    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4ltv\" (UniqueName: \"kubernetes.io/projected/d229854a-a04b-42fc-ace8-a5ac99979da8-kube-api-access-n4ltv\") pod \"d229854a-a04b-42fc-ace8-a5ac99979da8\" (UID: \"d229854a-a04b-42fc-ace8-a5ac99979da8\") "
Sep 17 23:31:10.540562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.540450    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-plugins-config-volume\") pod \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\" (UID: \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\") "
Sep 17 23:31:10.540562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.540486    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d229854a-a04b-42fc-ace8-a5ac99979da8-plugins-config-volume\") pod \"d229854a-a04b-42fc-ace8-a5ac99979da8\" (UID: \"d229854a-a04b-42fc-ace8-a5ac99979da8\") "
Sep 17 23:31:10.540562 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.540525    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdpgx\" (UniqueName: \"kubernetes.io/projected/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-kube-api-access-kdpgx\") pod \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\" (UID: \"a4b06b80-0e03-41f5-b00b-a131ae8e98cb\") "
Sep 17 23:31:10.540830 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.540806    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4b06b80-0e03-41f5-b00b-a131ae8e98cb" (UID: "a4b06b80-0e03-41f5-b00b-a131ae8e98cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:31:10.540928 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.540860    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d229854a-a04b-42fc-ace8-a5ac99979da8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d229854a-a04b-42fc-ace8-a5ac99979da8" (UID: "d229854a-a04b-42fc-ace8-a5ac99979da8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:31:10.540928 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.540901    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:10.542517 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.542495    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d229854a-a04b-42fc-ace8-a5ac99979da8-kube-api-access-n4ltv" (OuterVolumeSpecName: "kube-api-access-n4ltv") pod "d229854a-a04b-42fc-ace8-a5ac99979da8" (UID: "d229854a-a04b-42fc-ace8-a5ac99979da8"). InnerVolumeSpecName "kube-api-access-n4ltv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:31:10.542606 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.542587    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-kube-api-access-kdpgx" (OuterVolumeSpecName: "kube-api-access-kdpgx") pod "a4b06b80-0e03-41f5-b00b-a131ae8e98cb" (UID: "a4b06b80-0e03-41f5-b00b-a131ae8e98cb"). InnerVolumeSpecName "kube-api-access-kdpgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:31:10.642052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.642017    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d229854a-a04b-42fc-ace8-a5ac99979da8-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:10.642052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.642045    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdpgx\" (UniqueName: \"kubernetes.io/projected/a4b06b80-0e03-41f5-b00b-a131ae8e98cb-kube-api-access-kdpgx\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:10.642052 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.642056    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4ltv\" (UniqueName: \"kubernetes.io/projected/d229854a-a04b-42fc-ace8-a5ac99979da8-kube-api-access-n4ltv\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:10.860676 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.860646    2581 generic.go:356] "Generic (PLEG): container finished" podID="d229854a-a04b-42fc-ace8-a5ac99979da8" containerID="bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b" exitCode=0
Sep 17 23:31:10.860872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.860694    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"
Sep 17 23:31:10.860872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.860736    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv" event={"ID":"d229854a-a04b-42fc-ace8-a5ac99979da8","Type":"ContainerDied","Data":"bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b"}
Sep 17 23:31:10.860872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.860792    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv" event={"ID":"d229854a-a04b-42fc-ace8-a5ac99979da8","Type":"ContainerDied","Data":"e8e099ef84a6e4cf79adb62840bc5b97a1c9542d9f570adae5f72188657bde23"}
Sep 17 23:31:10.860872 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.860819    2581 scope.go:117] "RemoveContainer" containerID="bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b"
Sep 17 23:31:10.862224 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.862198    2581 generic.go:356] "Generic (PLEG): container finished" podID="a4b06b80-0e03-41f5-b00b-a131ae8e98cb" containerID="510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603" exitCode=0
Sep 17 23:31:10.862311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.862245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg" event={"ID":"a4b06b80-0e03-41f5-b00b-a131ae8e98cb","Type":"ContainerDied","Data":"510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603"}
Sep 17 23:31:10.862311 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.862294    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg" event={"ID":"a4b06b80-0e03-41f5-b00b-a131ae8e98cb","Type":"ContainerDied","Data":"2c604770ca43ccf4eebce65cac10643c9f30cdf5a29c6180621245621693f8e6"}
Sep 17 23:31:10.862405 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.862251    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"
Sep 17 23:31:10.870368 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.870348    2581 scope.go:117] "RemoveContainer" containerID="bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b"
Sep 17 23:31:10.870645 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:31:10.870625    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b\": container with ID starting with bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b not found: ID does not exist" containerID="bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b"
Sep 17 23:31:10.870702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.870653    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b"} err="failed to get container status \"bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b\": rpc error: code = NotFound desc = could not find container \"bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b\": container with ID starting with bafd3a2ec781eafabc0743d661cdbb1e376c2d5aa234b3358e35d5aac0dd559b not found: ID does not exist"
Sep 17 23:31:10.870702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.870668    2581 scope.go:117] "RemoveContainer" containerID="510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603"
Sep 17 23:31:10.880203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.880185    2581 scope.go:117] "RemoveContainer" containerID="510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603"
Sep 17 23:31:10.880472 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:31:10.880454    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603\": container with ID starting with 510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603 not found: ID does not exist" containerID="510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603"
Sep 17 23:31:10.880518 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.880479    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603"} err="failed to get container status \"510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603\": rpc error: code = NotFound desc = could not find container \"510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603\": container with ID starting with 510a1baa258ac4463a25a51219aab56a089fdb25ce2692df0028c478af0ee603 not found: ID does not exist"
Sep 17 23:31:10.893203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.892450    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"]
Sep 17 23:31:10.899179 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.899152    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-09c1c1-79bbb4d6-cttlg"]
Sep 17 23:31:10.915612 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.915588    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"]
Sep 17 23:31:10.918016 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:10.917995    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-09c1c1-5fd6d77787-7cnzv"]
Sep 17 23:31:11.994050 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:11.994019    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25cf16e7-a51b-4e40-aac3-6060e94cc265" path="/var/lib/kubelet/pods/25cf16e7-a51b-4e40-aac3-6060e94cc265/volumes"
Sep 17 23:31:11.994440 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:11.994388    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4b06b80-0e03-41f5-b00b-a131ae8e98cb" path="/var/lib/kubelet/pods/a4b06b80-0e03-41f5-b00b-a131ae8e98cb/volumes"
Sep 17 23:31:11.994688 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:11.994675    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b944ac16-da00-45b3-9bb0-08255e071de9" path="/var/lib/kubelet/pods/b944ac16-da00-45b3-9bb0-08255e071de9/volumes"
Sep 17 23:31:11.994996 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:11.994984    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d229854a-a04b-42fc-ace8-a5ac99979da8" path="/var/lib/kubelet/pods/d229854a-a04b-42fc-ace8-a5ac99979da8/volumes"
Sep 17 23:31:12.765948 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:12.765910    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"]
Sep 17 23:31:12.768565 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:12.768540    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-7ab633-29828130-xb4hn"]
Sep 17 23:31:13.993325 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:13.993295    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="518a9c6d-776e-4289-8a0a-dca6b1ad17db" path="/var/lib/kubelet/pods/518a9c6d-776e-4289-8a0a-dca6b1ad17db/volumes"
Sep 17 23:31:20.115579 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.115533    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn" podUID="87802f0c-c645-4ef9-8d8a-abb56d55d89e" containerName="main" containerID="cri-o://3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43" gracePeriod=45
Sep 17 23:31:20.365568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.365537    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"]
Sep 17 23:31:20.365821 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.365766    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy" containerID="cri-o://3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc" gracePeriod=30
Sep 17 23:31:20.383406 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.383380    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:31:20.440435 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440406    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-dshm\") pod \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") "
Sep 17 23:31:20.440865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440458    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-model-cache\") pod \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") "
Sep 17 23:31:20.440865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440515    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tmp-dir\") pod \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") "
Sep 17 23:31:20.440865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440553    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tls-certs\") pod \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") "
Sep 17 23:31:20.440865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440585    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-home\") pod \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") "
Sep 17 23:31:20.440865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440636    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntx5k\" (UniqueName: \"kubernetes.io/projected/87802f0c-c645-4ef9-8d8a-abb56d55d89e-kube-api-access-ntx5k\") pod \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\" (UID: \"87802f0c-c645-4ef9-8d8a-abb56d55d89e\") "
Sep 17 23:31:20.440865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440758    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-model-cache" (OuterVolumeSpecName: "model-cache") pod "87802f0c-c645-4ef9-8d8a-abb56d55d89e" (UID: "87802f0c-c645-4ef9-8d8a-abb56d55d89e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:20.440865 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440785    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "87802f0c-c645-4ef9-8d8a-abb56d55d89e" (UID: "87802f0c-c645-4ef9-8d8a-abb56d55d89e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:20.441352 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.440960    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-home" (OuterVolumeSpecName: "home") pod "87802f0c-c645-4ef9-8d8a-abb56d55d89e" (UID: "87802f0c-c645-4ef9-8d8a-abb56d55d89e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:20.441563 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.441524    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:20.441563 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.441544    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:20.441563 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.441558    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:20.443244 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.443197    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-dshm" (OuterVolumeSpecName: "dshm") pod "87802f0c-c645-4ef9-8d8a-abb56d55d89e" (UID: "87802f0c-c645-4ef9-8d8a-abb56d55d89e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:20.443347 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.443287    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87802f0c-c645-4ef9-8d8a-abb56d55d89e" (UID: "87802f0c-c645-4ef9-8d8a-abb56d55d89e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:31:20.443435 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.443408    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87802f0c-c645-4ef9-8d8a-abb56d55d89e-kube-api-access-ntx5k" (OuterVolumeSpecName: "kube-api-access-ntx5k") pod "87802f0c-c645-4ef9-8d8a-abb56d55d89e" (UID: "87802f0c-c645-4ef9-8d8a-abb56d55d89e"). InnerVolumeSpecName "kube-api-access-ntx5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:31:20.542667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.542633    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87802f0c-c645-4ef9-8d8a-abb56d55d89e-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:20.542667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.542662    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntx5k\" (UniqueName: \"kubernetes.io/projected/87802f0c-c645-4ef9-8d8a-abb56d55d89e-kube-api-access-ntx5k\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:20.542667 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.542671    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87802f0c-c645-4ef9-8d8a-abb56d55d89e-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:20.932960 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.932925    2581 generic.go:356] "Generic (PLEG): container finished" podID="87802f0c-c645-4ef9-8d8a-abb56d55d89e" containerID="3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43" exitCode=0
Sep 17 23:31:20.933193 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.932969    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"
Sep 17 23:31:20.933193 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.933009    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn" event={"ID":"87802f0c-c645-4ef9-8d8a-abb56d55d89e","Type":"ContainerDied","Data":"3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43"}
Sep 17 23:31:20.933193 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.933050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn" event={"ID":"87802f0c-c645-4ef9-8d8a-abb56d55d89e","Type":"ContainerDied","Data":"c12044a85a1bd7ea647b48d60dfb636bd3eae94c89efc167fae5c6b57a6c6f17"}
Sep 17 23:31:20.933193 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.933067    2581 scope.go:117] "RemoveContainer" containerID="3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43"
Sep 17 23:31:20.943456 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.943438    2581 scope.go:117] "RemoveContainer" containerID="3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43"
Sep 17 23:31:20.943757 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:31:20.943739    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43\": container with ID starting with 3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43 not found: ID does not exist" containerID="3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43"
Sep 17 23:31:20.943833 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.943769    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43"} err="failed to get container status \"3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43\": rpc error: code = NotFound desc = could not find container \"3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43\": container with ID starting with 3f7a0173408e2f8bbf1c4cdb4a344a949d349971c9d295917c9aec8a43a0fe43 not found: ID does not exist"
Sep 17 23:31:20.956328 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.956305    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"]
Sep 17 23:31:20.959871 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:20.959850    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-7ab633/premium-w4-7ab633-kserve-7bf496c678-v9dfn"]
Sep 17 23:31:21.994934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:21.994900    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87802f0c-c645-4ef9-8d8a-abb56d55d89e" path="/var/lib/kubelet/pods/87802f0c-c645-4ef9-8d8a-abb56d55d89e/volumes"
Sep 17 23:31:22.388376 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:22.388349    2581 scope.go:117] "RemoveContainer" containerID="3f9dfbc5914c26e76c76b598edd1f857eaa9beba48f3195dfe4e80767b90961a"
Sep 17 23:31:25.632022 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.631997    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:31:25.689176 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689081    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-certs\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689176 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689138    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-socket\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689176 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49509f48-becc-4a36-9b74-69b886c9f35e-istio-podinfo\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689450 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689218    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-envoy\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689450 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689264    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-istio-token\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689450 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689310    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49509f48-becc-4a36-9b74-69b886c9f35e-istiod-ca-cert\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689450 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689335    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-data\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689475    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xb5td\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-kube-api-access-xb5td\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689508    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-credential-socket\") pod \"49509f48-becc-4a36-9b74-69b886c9f35e\" (UID: \"49509f48-becc-4a36-9b74-69b886c9f35e\") "
Sep 17 23:31:25.689651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689468    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:25.689651 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689540    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:25.689868 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689661    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-data" (OuterVolumeSpecName: "istio-data") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:25.689868 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689808    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:25.689991 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689900    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.689991 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689948    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-workload-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.689991 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689967    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-data\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.689991 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.689980    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-credential-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.690223 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.690053    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49509f48-becc-4a36-9b74-69b886c9f35e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:31:25.691922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.691894    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49509f48-becc-4a36-9b74-69b886c9f35e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:31:25.692205 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.692180    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:31:25.697966 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.697940    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-kube-api-access-xb5td" (OuterVolumeSpecName: "kube-api-access-xb5td") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "kube-api-access-xb5td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:31:25.698045 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.697947    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-istio-token" (OuterVolumeSpecName: "istio-token") pod "49509f48-becc-4a36-9b74-69b886c9f35e" (UID: "49509f48-becc-4a36-9b74-69b886c9f35e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:31:25.791455 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.791419    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xb5td\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-kube-api-access-xb5td\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.791455 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.791449    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49509f48-becc-4a36-9b74-69b886c9f35e-istio-podinfo\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.791455 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.791459    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49509f48-becc-4a36-9b74-69b886c9f35e-istio-envoy\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.791455 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.791467    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49509f48-becc-4a36-9b74-69b886c9f35e-istio-token\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.791714 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.791475    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49509f48-becc-4a36-9b74-69b886c9f35e-istiod-ca-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:25.966049 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.965961    2581 generic.go:356] "Generic (PLEG): container finished" podID="49509f48-becc-4a36-9b74-69b886c9f35e" containerID="3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc" exitCode=0
Sep 17 23:31:25.966049 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.966011    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"
Sep 17 23:31:25.966049 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.966001    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" event={"ID":"49509f48-becc-4a36-9b74-69b886c9f35e","Type":"ContainerDied","Data":"3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc"}
Sep 17 23:31:25.966336 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.966093    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw" event={"ID":"49509f48-becc-4a36-9b74-69b886c9f35e","Type":"ContainerDied","Data":"505c041f22e8715c812e8e0f80fee839f7e68d6030039690a49a1aa3e1c7c484"}
Sep 17 23:31:25.966336 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.966106    2581 scope.go:117] "RemoveContainer" containerID="3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc"
Sep 17 23:31:25.975981 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.975959    2581 scope.go:117] "RemoveContainer" containerID="59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3"
Sep 17 23:31:25.984795 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.984775    2581 scope.go:117] "RemoveContainer" containerID="3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc"
Sep 17 23:31:25.985049 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:31:25.985031    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc\": container with ID starting with 3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc not found: ID does not exist" containerID="3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc"
Sep 17 23:31:25.985106 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.985057    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc"} err="failed to get container status \"3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc\": rpc error: code = NotFound desc = could not find container \"3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc\": container with ID starting with 3fcc1084c6b17b789ffab07a94ef43aa4b207fad2f0be29e4dcab3fba7902abc not found: ID does not exist"
Sep 17 23:31:25.985106 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.985075    2581 scope.go:117] "RemoveContainer" containerID="59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3"
Sep 17 23:31:25.985327 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:31:25.985309    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3\": container with ID starting with 59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3 not found: ID does not exist" containerID="59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3"
Sep 17 23:31:25.985371 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.985333    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3"} err="failed to get container status \"59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3\": rpc error: code = NotFound desc = could not find container \"59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3\": container with ID starting with 59a613e4b19817e9cc93312d012616a7c301cd04112a7970db17b613b9d118b3 not found: ID does not exist"
Sep 17 23:31:25.994768 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.994748    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"]
Sep 17 23:31:25.996347 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:25.996324    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-7ab633-openshift-default-d68f89768-cqlmw"]
Sep 17 23:31:27.895883 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:27.895840    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"]
Sep 17 23:31:27.896470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:27.896148    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd" podUID="f6e4f647-6569-4236-aabb-ebd6aab162f2" containerName="payload-pre-processing" containerID="cri-o://fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96" gracePeriod=30
Sep 17 23:31:27.993234 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:27.993206    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" path="/var/lib/kubelet/pods/49509f48-becc-4a36-9b74-69b886c9f35e/volumes"
Sep 17 23:31:28.149778 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.149719    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:31:28.215965 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.215931    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wr92\" (UniqueName: \"kubernetes.io/projected/f6e4f647-6569-4236-aabb-ebd6aab162f2-kube-api-access-4wr92\") pod \"f6e4f647-6569-4236-aabb-ebd6aab162f2\" (UID: \"f6e4f647-6569-4236-aabb-ebd6aab162f2\") "
Sep 17 23:31:28.216191 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.216039    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e4f647-6569-4236-aabb-ebd6aab162f2-plugins-config-volume\") pod \"f6e4f647-6569-4236-aabb-ebd6aab162f2\" (UID: \"f6e4f647-6569-4236-aabb-ebd6aab162f2\") "
Sep 17 23:31:28.216469 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.216440    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e4f647-6569-4236-aabb-ebd6aab162f2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6e4f647-6569-4236-aabb-ebd6aab162f2" (UID: "f6e4f647-6569-4236-aabb-ebd6aab162f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:31:28.219305 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.219256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e4f647-6569-4236-aabb-ebd6aab162f2-kube-api-access-4wr92" (OuterVolumeSpecName: "kube-api-access-4wr92") pod "f6e4f647-6569-4236-aabb-ebd6aab162f2" (UID: "f6e4f647-6569-4236-aabb-ebd6aab162f2"). InnerVolumeSpecName "kube-api-access-4wr92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:31:28.317372 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.317335    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6e4f647-6569-4236-aabb-ebd6aab162f2-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:28.317372 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.317366    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wr92\" (UniqueName: \"kubernetes.io/projected/f6e4f647-6569-4236-aabb-ebd6aab162f2-kube-api-access-4wr92\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:31:28.987470 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.987439    2581 generic.go:356] "Generic (PLEG): container finished" podID="f6e4f647-6569-4236-aabb-ebd6aab162f2" containerID="fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96" exitCode=0
Sep 17 23:31:28.987944 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.987484    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"
Sep 17 23:31:28.987944 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.987525    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd" event={"ID":"f6e4f647-6569-4236-aabb-ebd6aab162f2","Type":"ContainerDied","Data":"fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96"}
Sep 17 23:31:28.987944 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.987584    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd" event={"ID":"f6e4f647-6569-4236-aabb-ebd6aab162f2","Type":"ContainerDied","Data":"294f0519043981657168711190701420bceca0c21511219c3a8eadfaa6e6edb0"}
Sep 17 23:31:28.987944 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:28.987619    2581 scope.go:117] "RemoveContainer" containerID="fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96"
Sep 17 23:31:29.000489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:29.000465    2581 scope.go:117] "RemoveContainer" containerID="fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96"
Sep 17 23:31:29.000802 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:31:29.000780    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96\": container with ID starting with fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96 not found: ID does not exist" containerID="fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96"
Sep 17 23:31:29.000868 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:29.000809    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96"} err="failed to get container status \"fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96\": rpc error: code = NotFound desc = could not find container \"fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96\": container with ID starting with fb444e308d77c76bc5ce950421588801962f951cd4d7daddc416bba7c7636b96 not found: ID does not exist"
Sep 17 23:31:29.011316 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:29.011289    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"]
Sep 17 23:31:29.015116 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:29.015094    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-55130c-7c969456c4-hr8pd"]
Sep 17 23:31:29.994874 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:31:29.994834    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e4f647-6569-4236-aabb-ebd6aab162f2" path="/var/lib/kubelet/pods/f6e4f647-6569-4236-aabb-ebd6aab162f2/volumes"
Sep 17 23:32:07.983602 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:07.983557    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"]
Sep 17 23:32:23.005634 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.005577    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4" podUID="33b3ce28-cfaf-4622-871e-97e01dd54f9f" containerName="main" containerID="cri-o://432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f" gracePeriod=45
Sep 17 23:32:23.276313 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.276290    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:32:23.324666 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.324625    2581 generic.go:356] "Generic (PLEG): container finished" podID="33b3ce28-cfaf-4622-871e-97e01dd54f9f" containerID="432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f" exitCode=0
Sep 17 23:32:23.324666 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.324663    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"
Sep 17 23:32:23.324906 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.324705    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4" event={"ID":"33b3ce28-cfaf-4622-871e-97e01dd54f9f","Type":"ContainerDied","Data":"432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f"}
Sep 17 23:32:23.324906 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.324755    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4" event={"ID":"33b3ce28-cfaf-4622-871e-97e01dd54f9f","Type":"ContainerDied","Data":"f3b8fcf1c6f2badc918ee5c18f50d21b842c65b90620182a127f51d4f465ebb2"}
Sep 17 23:32:23.324906 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.324776    2581 scope.go:117] "RemoveContainer" containerID="432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f"
Sep 17 23:32:23.333704 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.333684    2581 scope.go:117] "RemoveContainer" containerID="432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f"
Sep 17 23:32:23.334004 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:32:23.333977    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f\": container with ID starting with 432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f not found: ID does not exist" containerID="432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f"
Sep 17 23:32:23.334069 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.334013    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f"} err="failed to get container status \"432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f\": rpc error: code = NotFound desc = could not find container \"432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f\": container with ID starting with 432ec4fdc6b5a591b07e6f9ce6fb3dab8445cced4869946c97218949d9b88e2f not found: ID does not exist"
Sep 17 23:32:23.443326 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443293    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-model-cache\") pod \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") "
Sep 17 23:32:23.443509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443368    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tls-certs\") pod \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") "
Sep 17 23:32:23.443509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443406    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tmp-dir\") pod \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") "
Sep 17 23:32:23.443509 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443437    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9f9z\" (UniqueName: \"kubernetes.io/projected/33b3ce28-cfaf-4622-871e-97e01dd54f9f-kube-api-access-h9f9z\") pod \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") "
Sep 17 23:32:23.443681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443544    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-dshm\") pod \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") "
Sep 17 23:32:23.443681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443600    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-home\") pod \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\" (UID: \"33b3ce28-cfaf-4622-871e-97e01dd54f9f\") "
Sep 17 23:32:23.443681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443606    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-model-cache" (OuterVolumeSpecName: "model-cache") pod "33b3ce28-cfaf-4622-871e-97e01dd54f9f" (UID: "33b3ce28-cfaf-4622-871e-97e01dd54f9f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:32:23.443833 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33b3ce28-cfaf-4622-871e-97e01dd54f9f" (UID: "33b3ce28-cfaf-4622-871e-97e01dd54f9f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:32:23.443833 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443825    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-home" (OuterVolumeSpecName: "home") pod "33b3ce28-cfaf-4622-871e-97e01dd54f9f" (UID: "33b3ce28-cfaf-4622-871e-97e01dd54f9f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:32:23.443988 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443969    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-model-cache\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:32:23.444068 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443990    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tmp-dir\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:32:23.444068 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.443999    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-home\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:32:23.445770 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.445738    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-dshm" (OuterVolumeSpecName: "dshm") pod "33b3ce28-cfaf-4622-871e-97e01dd54f9f" (UID: "33b3ce28-cfaf-4622-871e-97e01dd54f9f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:32:23.445878 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.445780    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b3ce28-cfaf-4622-871e-97e01dd54f9f-kube-api-access-h9f9z" (OuterVolumeSpecName: "kube-api-access-h9f9z") pod "33b3ce28-cfaf-4622-871e-97e01dd54f9f" (UID: "33b3ce28-cfaf-4622-871e-97e01dd54f9f"). InnerVolumeSpecName "kube-api-access-h9f9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:32:23.445878 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.445802    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33b3ce28-cfaf-4622-871e-97e01dd54f9f" (UID: "33b3ce28-cfaf-4622-871e-97e01dd54f9f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:32:23.545540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.545453    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33b3ce28-cfaf-4622-871e-97e01dd54f9f-tls-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:32:23.545540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.545483    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9f9z\" (UniqueName: \"kubernetes.io/projected/33b3ce28-cfaf-4622-871e-97e01dd54f9f-kube-api-access-h9f9z\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:32:23.545540 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.545497    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33b3ce28-cfaf-4622-871e-97e01dd54f9f-dshm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:32:23.647518 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.647489    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"]
Sep 17 23:32:23.651589 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.651559    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e5aae7/auth-test-model-e5aae7-kserve-7b6f6cfc5-284f4"]
Sep 17 23:32:23.993554 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:32:23.993523    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33b3ce28-cfaf-4622-871e-97e01dd54f9f" path="/var/lib/kubelet/pods/33b3ce28-cfaf-4622-871e-97e01dd54f9f/volumes"
Sep 17 23:34:46.991841 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.991751    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"]
Sep 17 23:34:46.992431 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992354    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87802f0c-c645-4ef9-8d8a-abb56d55d89e" containerName="main"
Sep 17 23:34:46.992431 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992376    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="87802f0c-c645-4ef9-8d8a-abb56d55d89e" containerName="main"
Sep 17 23:34:46.992431 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992416    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4b06b80-0e03-41f5-b00b-a131ae8e98cb" containerName="payload-pre-processing"
Sep 17 23:34:46.992431 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992428    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4b06b80-0e03-41f5-b00b-a131ae8e98cb" containerName="payload-pre-processing"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992439    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d229854a-a04b-42fc-ace8-a5ac99979da8" containerName="payload-processing"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992447    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d229854a-a04b-42fc-ace8-a5ac99979da8" containerName="payload-processing"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992458    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e4f647-6569-4236-aabb-ebd6aab162f2" containerName="payload-pre-processing"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992466    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e4f647-6569-4236-aabb-ebd6aab162f2" containerName="payload-pre-processing"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992481    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33b3ce28-cfaf-4622-871e-97e01dd54f9f" containerName="main"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992489    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="33b3ce28-cfaf-4622-871e-97e01dd54f9f" containerName="main"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992507    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992515    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992529    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992537    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992554    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b944ac16-da00-45b3-9bb0-08255e071de9" containerName="revoke-keys"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992562    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b944ac16-da00-45b3-9bb0-08255e071de9" containerName="revoke-keys"
Sep 17 23:34:46.992658 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992657    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy"
Sep 17 23:34:46.993263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992670    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="33b3ce28-cfaf-4622-871e-97e01dd54f9f" containerName="main"
Sep 17 23:34:46.993263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992679    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="49509f48-becc-4a36-9b74-69b886c9f35e" containerName="istio-proxy"
Sep 17 23:34:46.993263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992689    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4b06b80-0e03-41f5-b00b-a131ae8e98cb" containerName="payload-pre-processing"
Sep 17 23:34:46.993263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992699    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e4f647-6569-4236-aabb-ebd6aab162f2" containerName="payload-pre-processing"
Sep 17 23:34:46.993263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992711    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d229854a-a04b-42fc-ace8-a5ac99979da8" containerName="payload-processing"
Sep 17 23:34:46.993263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992719    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="87802f0c-c645-4ef9-8d8a-abb56d55d89e" containerName="main"
Sep 17 23:34:46.993263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.992728    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b944ac16-da00-45b3-9bb0-08255e071de9" containerName="revoke-keys"
Sep 17 23:34:46.998201 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:46.997248    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:47.000922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.000898    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-93a496-dockercfg-9s4tc\""
Sep 17 23:34:47.001037 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.000945    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-93a496\""
Sep 17 23:34:47.008108 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.007373    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"]
Sep 17 23:34:47.109095 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.109059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8dss\" (UniqueName: \"kubernetes.io/projected/dfa62470-4985-41b1-b021-17b9e5afd15d-kube-api-access-z8dss\") pod \"payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm\" (UID: \"dfa62470-4985-41b1-b021-17b9e5afd15d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:47.109095 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.109100    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa62470-4985-41b1-b021-17b9e5afd15d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm\" (UID: \"dfa62470-4985-41b1-b021-17b9e5afd15d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:47.210569 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.210530    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8dss\" (UniqueName: \"kubernetes.io/projected/dfa62470-4985-41b1-b021-17b9e5afd15d-kube-api-access-z8dss\") pod \"payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm\" (UID: \"dfa62470-4985-41b1-b021-17b9e5afd15d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:47.210569 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.210569    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa62470-4985-41b1-b021-17b9e5afd15d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm\" (UID: \"dfa62470-4985-41b1-b021-17b9e5afd15d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:47.211274 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.211255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa62470-4985-41b1-b021-17b9e5afd15d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm\" (UID: \"dfa62470-4985-41b1-b021-17b9e5afd15d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:47.219263 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.219236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8dss\" (UniqueName: \"kubernetes.io/projected/dfa62470-4985-41b1-b021-17b9e5afd15d-kube-api-access-z8dss\") pod \"payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm\" (UID: \"dfa62470-4985-41b1-b021-17b9e5afd15d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:47.312192 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.312078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:47.586211 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:47.586183    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"]
Sep 17 23:34:47.586994 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:34:47.586970    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa62470_4985_41b1_b021_17b9e5afd15d.slice/crio-06956bc69126e80846b7b77aa86ff34b217f1c40475275561e96d17ee00516cb WatchSource:0}: Error finding container 06956bc69126e80846b7b77aa86ff34b217f1c40475275561e96d17ee00516cb: Status 404 returned error can't find the container with id 06956bc69126e80846b7b77aa86ff34b217f1c40475275561e96d17ee00516cb
Sep 17 23:34:48.183141 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:48.183103    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm" event={"ID":"dfa62470-4985-41b1-b021-17b9e5afd15d","Type":"ContainerStarted","Data":"b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23"}
Sep 17 23:34:48.183522 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:48.183179    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm" event={"ID":"dfa62470-4985-41b1-b021-17b9e5afd15d","Type":"ContainerStarted","Data":"06956bc69126e80846b7b77aa86ff34b217f1c40475275561e96d17ee00516cb"}
Sep 17 23:34:48.183522 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:48.183205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:34:48.200659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:48.200611    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm" podStartSLOduration=2.200596706 podStartE2EDuration="2.200596706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:34:48.199251765 +0000 UTC m=+1708.203443608" watchObservedRunningTime="2026-09-17 23:34:48.200596706 +0000 UTC m=+1708.204788545"
Sep 17 23:34:59.192258 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:34:59.192224    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:35:16.295635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.295595    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"]
Sep 17 23:35:16.299718 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.299694    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:16.302449 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.302427    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-86dda1-dockercfg-p86j4\""
Sep 17 23:35:16.302584 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.302428    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-86dda1\""
Sep 17 23:35:16.309267 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.309240    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"]
Sep 17 23:35:16.380418 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.380377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqdps\" (UniqueName: \"kubernetes.io/projected/d68b5355-4867-481f-b8c3-e274bdbe6d13-kube-api-access-lqdps\") pod \"payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd\" (UID: \"d68b5355-4867-481f-b8c3-e274bdbe6d13\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:16.380588 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.380534    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68b5355-4867-481f-b8c3-e274bdbe6d13-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd\" (UID: \"d68b5355-4867-481f-b8c3-e274bdbe6d13\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:16.481861 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.481825    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqdps\" (UniqueName: \"kubernetes.io/projected/d68b5355-4867-481f-b8c3-e274bdbe6d13-kube-api-access-lqdps\") pod \"payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd\" (UID: \"d68b5355-4867-481f-b8c3-e274bdbe6d13\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:16.482056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.481897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68b5355-4867-481f-b8c3-e274bdbe6d13-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd\" (UID: \"d68b5355-4867-481f-b8c3-e274bdbe6d13\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:16.482680 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.482654    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68b5355-4867-481f-b8c3-e274bdbe6d13-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd\" (UID: \"d68b5355-4867-481f-b8c3-e274bdbe6d13\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:16.490312 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.490288    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqdps\" (UniqueName: \"kubernetes.io/projected/d68b5355-4867-481f-b8c3-e274bdbe6d13-kube-api-access-lqdps\") pod \"payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd\" (UID: \"d68b5355-4867-481f-b8c3-e274bdbe6d13\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:16.612047 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.612013    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:16.891280 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:16.891249    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"]
Sep 17 23:35:16.892783 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:35:16.892752    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68b5355_4867_481f_b8c3_e274bdbe6d13.slice/crio-b7b29fa851ddd13fec7cf2762c00e4b972ea8b3ec80ce390f08c6b5a3bf7ee57 WatchSource:0}: Error finding container b7b29fa851ddd13fec7cf2762c00e4b972ea8b3ec80ce390f08c6b5a3bf7ee57: Status 404 returned error can't find the container with id b7b29fa851ddd13fec7cf2762c00e4b972ea8b3ec80ce390f08c6b5a3bf7ee57
Sep 17 23:35:17.357702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:17.357621    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd" event={"ID":"d68b5355-4867-481f-b8c3-e274bdbe6d13","Type":"ContainerStarted","Data":"c39b73304807d4f71cfeb21a7b17b50645a64cbbb76fa675fc072aa2ef5039d2"}
Sep 17 23:35:17.357702 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:17.357672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd" event={"ID":"d68b5355-4867-481f-b8c3-e274bdbe6d13","Type":"ContainerStarted","Data":"b7b29fa851ddd13fec7cf2762c00e4b972ea8b3ec80ce390f08c6b5a3bf7ee57"}
Sep 17 23:35:17.358212 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:17.357719    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:17.376803 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:17.376757    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd" podStartSLOduration=1.376741237 podStartE2EDuration="1.376741237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:35:17.374459166 +0000 UTC m=+1737.378651005" watchObservedRunningTime="2026-09-17 23:35:17.376741237 +0000 UTC m=+1737.380933114"
Sep 17 23:35:28.366284 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:28.366248    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:35:43.268461 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:43.268435    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-kd6gq_9babfa47-30fb-4a73-88f9-8a090d496b96/payload-processing/0.log"
Sep 17 23:35:43.385815 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:35:43.385782    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd_d68b5355-4867-481f-b8c3-e274bdbe6d13/payload-processing/0.log"
Sep 17 23:36:19.372228 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:19.372194    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm_dfa62470-4985-41b1-b021-17b9e5afd15d/payload-processing/0.log"
Sep 17 23:36:19.491430 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:19.491402    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-kd6gq_9babfa47-30fb-4a73-88f9-8a090d496b96/payload-processing/0.log"
Sep 17 23:36:19.613583 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:19.613558    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd_d68b5355-4867-481f-b8c3-e274bdbe6d13/payload-processing/0.log"
Sep 17 23:36:19.731871 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:19.731795    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd_d68b5355-4867-481f-b8c3-e274bdbe6d13/payload-processing/0.log"
Sep 17 23:36:22.497734 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:22.497686    2581 scope.go:117] "RemoveContainer" containerID="bf023fc17d2d146b1e257e1ca2c5c827988d317d593aeee57f30abc28afadf1c"
Sep 17 23:36:22.506968 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:22.506949    2581 scope.go:117] "RemoveContainer" containerID="9574dd9d1866f622868cba4375dd98c48dc91a991878af2d6a95c6e3f9687ef4"
Sep 17 23:36:22.515253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:22.515235    2581 scope.go:117] "RemoveContainer" containerID="d71c26cb2fee89121c3da1d9cf3d8db41c08da410c22d4a15095871d4ec6813d"
Sep 17 23:36:22.523375 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:22.523352    2581 scope.go:117] "RemoveContainer" containerID="9b32f96be1bd003e8a985927686b6fc53d62639ed6932dd7b83f593761b2ca0c"
Sep 17 23:36:22.533262 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:22.533246    2581 scope.go:117] "RemoveContainer" containerID="a93f09b92fe918a3d79554a85c4fe857db9110b032eff09461c08d6605f10790"
Sep 17 23:36:27.560676 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.560638    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"]
Sep 17 23:36:27.566329 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.566304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:27.569001 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.568973    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-145e15-dockercfg-pdn7s\""
Sep 17 23:36:27.569156 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.569022    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-145e15\""
Sep 17 23:36:27.577240 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.577216    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"]
Sep 17 23:36:27.642563 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.642525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2458q\" (UniqueName: \"kubernetes.io/projected/0145a534-37df-49b4-829d-77028c8328a3-kube-api-access-2458q\") pod \"payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872\" (UID: \"0145a534-37df-49b4-829d-77028c8328a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:27.642789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.642751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0145a534-37df-49b4-829d-77028c8328a3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872\" (UID: \"0145a534-37df-49b4-829d-77028c8328a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:27.743942 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.743909    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0145a534-37df-49b4-829d-77028c8328a3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872\" (UID: \"0145a534-37df-49b4-829d-77028c8328a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:27.744117 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.743963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2458q\" (UniqueName: \"kubernetes.io/projected/0145a534-37df-49b4-829d-77028c8328a3-kube-api-access-2458q\") pod \"payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872\" (UID: \"0145a534-37df-49b4-829d-77028c8328a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:27.744634 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.744612    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0145a534-37df-49b4-829d-77028c8328a3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872\" (UID: \"0145a534-37df-49b4-829d-77028c8328a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:27.752617 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.752586    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2458q\" (UniqueName: \"kubernetes.io/projected/0145a534-37df-49b4-829d-77028c8328a3-kube-api-access-2458q\") pod \"payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872\" (UID: \"0145a534-37df-49b4-829d-77028c8328a3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:27.881019 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:27.880984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:28.162091 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:28.162061    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"]
Sep 17 23:36:28.163850 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:36:28.163809    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0145a534_37df_49b4_829d_77028c8328a3.slice/crio-251c4d289546a2bde94c5bcc49827e91f51327f6e8e3d053125fbd01bbb5292f WatchSource:0}: Error finding container 251c4d289546a2bde94c5bcc49827e91f51327f6e8e3d053125fbd01bbb5292f: Status 404 returned error can't find the container with id 251c4d289546a2bde94c5bcc49827e91f51327f6e8e3d053125fbd01bbb5292f
Sep 17 23:36:28.782021 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:28.781976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872" event={"ID":"0145a534-37df-49b4-829d-77028c8328a3","Type":"ContainerStarted","Data":"fb16bdef13d048731db954cf3c1b6ad96292c24ed968074d544bdf13b110d60f"}
Sep 17 23:36:28.782474 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:28.782037    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872" event={"ID":"0145a534-37df-49b4-829d-77028c8328a3","Type":"ContainerStarted","Data":"251c4d289546a2bde94c5bcc49827e91f51327f6e8e3d053125fbd01bbb5292f"}
Sep 17 23:36:28.782474 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:28.782064    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:28.800807 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:28.800760    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872" podStartSLOduration=1.8007448799999999 podStartE2EDuration="1.80074488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:36:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:36:28.798155662 +0000 UTC m=+1808.802347505" watchObservedRunningTime="2026-09-17 23:36:28.80074488 +0000 UTC m=+1808.804936718"
Sep 17 23:36:39.790766 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:39.790734    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:55.722789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:55.722754    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"]
Sep 17 23:36:55.723369 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:55.723006    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872" podUID="0145a534-37df-49b4-829d-77028c8328a3" containerName="payload-processing" containerID="cri-o://fb16bdef13d048731db954cf3c1b6ad96292c24ed968074d544bdf13b110d60f" gracePeriod=30
Sep 17 23:36:55.940549 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:55.940514    2581 generic.go:356] "Generic (PLEG): container finished" podID="0145a534-37df-49b4-829d-77028c8328a3" containerID="fb16bdef13d048731db954cf3c1b6ad96292c24ed968074d544bdf13b110d60f" exitCode=0
Sep 17 23:36:55.940703 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:55.940544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872" event={"ID":"0145a534-37df-49b4-829d-77028c8328a3","Type":"ContainerDied","Data":"fb16bdef13d048731db954cf3c1b6ad96292c24ed968074d544bdf13b110d60f"}
Sep 17 23:36:55.986290 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:55.986234    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:56.116085 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.116047    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2458q\" (UniqueName: \"kubernetes.io/projected/0145a534-37df-49b4-829d-77028c8328a3-kube-api-access-2458q\") pod \"0145a534-37df-49b4-829d-77028c8328a3\" (UID: \"0145a534-37df-49b4-829d-77028c8328a3\") "
Sep 17 23:36:56.116274 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.116164    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0145a534-37df-49b4-829d-77028c8328a3-plugins-config-volume\") pod \"0145a534-37df-49b4-829d-77028c8328a3\" (UID: \"0145a534-37df-49b4-829d-77028c8328a3\") "
Sep 17 23:36:56.116530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.116506    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0145a534-37df-49b4-829d-77028c8328a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0145a534-37df-49b4-829d-77028c8328a3" (UID: "0145a534-37df-49b4-829d-77028c8328a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:36:56.118177 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.118156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0145a534-37df-49b4-829d-77028c8328a3-kube-api-access-2458q" (OuterVolumeSpecName: "kube-api-access-2458q") pod "0145a534-37df-49b4-829d-77028c8328a3" (UID: "0145a534-37df-49b4-829d-77028c8328a3"). InnerVolumeSpecName "kube-api-access-2458q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:36:56.217571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.217533    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2458q\" (UniqueName: \"kubernetes.io/projected/0145a534-37df-49b4-829d-77028c8328a3-kube-api-access-2458q\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:36:56.217571 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.217564    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0145a534-37df-49b4-829d-77028c8328a3-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:36:56.947531 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.947499    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"
Sep 17 23:36:56.947531 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.947515    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872" event={"ID":"0145a534-37df-49b4-829d-77028c8328a3","Type":"ContainerDied","Data":"251c4d289546a2bde94c5bcc49827e91f51327f6e8e3d053125fbd01bbb5292f"}
Sep 17 23:36:56.948062 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.947596    2581 scope.go:117] "RemoveContainer" containerID="fb16bdef13d048731db954cf3c1b6ad96292c24ed968074d544bdf13b110d60f"
Sep 17 23:36:56.971735 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.971709    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"]
Sep 17 23:36:56.977015 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:56.976991    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-145e15-7db5cb9b9c-jw872"]
Sep 17 23:36:57.993845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:36:57.993810    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0145a534-37df-49b4-829d-77028c8328a3" path="/var/lib/kubelet/pods/0145a534-37df-49b4-829d-77028c8328a3/volumes"
Sep 17 23:37:05.823314 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.823277    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"]
Sep 17 23:37:05.823712 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.823684    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0145a534-37df-49b4-829d-77028c8328a3" containerName="payload-processing"
Sep 17 23:37:05.823712 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.823695    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0145a534-37df-49b4-829d-77028c8328a3" containerName="payload-processing"
Sep 17 23:37:05.823797 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.823766    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0145a534-37df-49b4-829d-77028c8328a3" containerName="payload-processing"
Sep 17 23:37:05.828104 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.828085    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"
Sep 17 23:37:05.830998 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.830973    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 23:37:05.830998 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.830974    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:37:05.836299 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.835913    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"]
Sep 17 23:37:05.903770 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:05.903738    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdd775-6f56-46c8-8a14-104bfe7a33c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-93a496-627rz\" (UID: \"e8bdd775-6f56-46c8-8a14-104bfe7a33c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"
Sep 17 23:37:06.004845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:06.004812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdd775-6f56-46c8-8a14-104bfe7a33c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-93a496-627rz\" (UID: \"e8bdd775-6f56-46c8-8a14-104bfe7a33c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"
Sep 17 23:37:06.005446 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:06.005428    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdd775-6f56-46c8-8a14-104bfe7a33c4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-93a496-627rz\" (UID: \"e8bdd775-6f56-46c8-8a14-104bfe7a33c4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"
Sep 17 23:37:06.140236 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:06.140199    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"
Sep 17 23:37:06.417933 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:06.417906    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"]
Sep 17 23:37:06.420058 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:37:06.420028    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bdd775_6f56_46c8_8a14_104bfe7a33c4.slice/crio-47d6d9ae11f659d6df3f581d30fcdb3e7fe6e0035f77b82997e37512ccac2765 WatchSource:0}: Error finding container 47d6d9ae11f659d6df3f581d30fcdb3e7fe6e0035f77b82997e37512ccac2765: Status 404 returned error can't find the container with id 47d6d9ae11f659d6df3f581d30fcdb3e7fe6e0035f77b82997e37512ccac2765
Sep 17 23:37:07.008742 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:07.008650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz" event={"ID":"e8bdd775-6f56-46c8-8a14-104bfe7a33c4","Type":"ContainerStarted","Data":"41baecea5777c5132d55557adb89df7a88e98d7d4f05ae8b406020ce41c2888f"}
Sep 17 23:37:07.008742 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:07.008704    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz" event={"ID":"e8bdd775-6f56-46c8-8a14-104bfe7a33c4","Type":"ContainerStarted","Data":"47d6d9ae11f659d6df3f581d30fcdb3e7fe6e0035f77b82997e37512ccac2765"}
Sep 17 23:37:07.025410 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:07.025363    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz" podStartSLOduration=2.025347773 podStartE2EDuration="2.025347773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:37:07.024373045 +0000 UTC m=+1847.028564898" watchObservedRunningTime="2026-09-17 23:37:07.025347773 +0000 UTC m=+1847.029539618"
Sep 17 23:37:08.015992 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:08.015960    2581 generic.go:356] "Generic (PLEG): container finished" podID="e8bdd775-6f56-46c8-8a14-104bfe7a33c4" containerID="41baecea5777c5132d55557adb89df7a88e98d7d4f05ae8b406020ce41c2888f" exitCode=0
Sep 17 23:37:08.016456 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:08.016046    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz" event={"ID":"e8bdd775-6f56-46c8-8a14-104bfe7a33c4","Type":"ContainerDied","Data":"41baecea5777c5132d55557adb89df7a88e98d7d4f05ae8b406020ce41c2888f"}
Sep 17 23:37:09.155966 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:09.155943    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"
Sep 17 23:37:09.239106 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:09.239072    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdd775-6f56-46c8-8a14-104bfe7a33c4-maas-api-service-ca\") pod \"e8bdd775-6f56-46c8-8a14-104bfe7a33c4\" (UID: \"e8bdd775-6f56-46c8-8a14-104bfe7a33c4\") "
Sep 17 23:37:09.239450 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:09.239428    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8bdd775-6f56-46c8-8a14-104bfe7a33c4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8bdd775-6f56-46c8-8a14-104bfe7a33c4" (UID: "e8bdd775-6f56-46c8-8a14-104bfe7a33c4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:37:09.340425 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:09.340338    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8bdd775-6f56-46c8-8a14-104bfe7a33c4-maas-api-service-ca\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:37:10.029454 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:10.029427    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"
Sep 17 23:37:10.029637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:10.029464    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz" event={"ID":"e8bdd775-6f56-46c8-8a14-104bfe7a33c4","Type":"ContainerDied","Data":"47d6d9ae11f659d6df3f581d30fcdb3e7fe6e0035f77b82997e37512ccac2765"}
Sep 17 23:37:10.029637 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:10.029509    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47d6d9ae11f659d6df3f581d30fcdb3e7fe6e0035f77b82997e37512ccac2765"
Sep 17 23:37:21.400084 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.400041    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"]
Sep 17 23:37:21.400521 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.400263    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm" podUID="dfa62470-4985-41b1-b021-17b9e5afd15d" containerName="payload-processing" containerID="cri-o://b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23" gracePeriod=30
Sep 17 23:37:21.581791 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.581741    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"]
Sep 17 23:37:21.586654 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.586625    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-93a496-627rz"]
Sep 17 23:37:21.664360 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.664300    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:37:21.756043 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.755998    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8dss\" (UniqueName: \"kubernetes.io/projected/dfa62470-4985-41b1-b021-17b9e5afd15d-kube-api-access-z8dss\") pod \"dfa62470-4985-41b1-b021-17b9e5afd15d\" (UID: \"dfa62470-4985-41b1-b021-17b9e5afd15d\") "
Sep 17 23:37:21.756235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.756201    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa62470-4985-41b1-b021-17b9e5afd15d-plugins-config-volume\") pod \"dfa62470-4985-41b1-b021-17b9e5afd15d\" (UID: \"dfa62470-4985-41b1-b021-17b9e5afd15d\") "
Sep 17 23:37:21.756590 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.756551    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfa62470-4985-41b1-b021-17b9e5afd15d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dfa62470-4985-41b1-b021-17b9e5afd15d" (UID: "dfa62470-4985-41b1-b021-17b9e5afd15d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:37:21.758402 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.758377    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa62470-4985-41b1-b021-17b9e5afd15d-kube-api-access-z8dss" (OuterVolumeSpecName: "kube-api-access-z8dss") pod "dfa62470-4985-41b1-b021-17b9e5afd15d" (UID: "dfa62470-4985-41b1-b021-17b9e5afd15d"). InnerVolumeSpecName "kube-api-access-z8dss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:37:21.857678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.857636    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa62470-4985-41b1-b021-17b9e5afd15d-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:37:21.857678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.857671    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8dss\" (UniqueName: \"kubernetes.io/projected/dfa62470-4985-41b1-b021-17b9e5afd15d-kube-api-access-z8dss\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:37:21.994597 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:21.994520    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8bdd775-6f56-46c8-8a14-104bfe7a33c4" path="/var/lib/kubelet/pods/e8bdd775-6f56-46c8-8a14-104bfe7a33c4/volumes"
Sep 17 23:37:22.105502 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.105469    2581 generic.go:356] "Generic (PLEG): container finished" podID="dfa62470-4985-41b1-b021-17b9e5afd15d" containerID="b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23" exitCode=0
Sep 17 23:37:22.105694 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.105526    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"
Sep 17 23:37:22.105694 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.105559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm" event={"ID":"dfa62470-4985-41b1-b021-17b9e5afd15d","Type":"ContainerDied","Data":"b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23"}
Sep 17 23:37:22.105694 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.105615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm" event={"ID":"dfa62470-4985-41b1-b021-17b9e5afd15d","Type":"ContainerDied","Data":"06956bc69126e80846b7b77aa86ff34b217f1c40475275561e96d17ee00516cb"}
Sep 17 23:37:22.105694 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.105639    2581 scope.go:117] "RemoveContainer" containerID="b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23"
Sep 17 23:37:22.114965 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.114943    2581 scope.go:117] "RemoveContainer" containerID="b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23"
Sep 17 23:37:22.115235 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:37:22.115216    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23\": container with ID starting with b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23 not found: ID does not exist" containerID="b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23"
Sep 17 23:37:22.115295 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.115243    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23"} err="failed to get container status \"b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23\": rpc error: code = NotFound desc = could not find container \"b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23\": container with ID starting with b1e92bd9c35c04e08372e1c5a4f16e1750db0f29b750f2ad576dbed7c49c2e23 not found: ID does not exist"
Sep 17 23:37:22.123482 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.123456    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"]
Sep 17 23:37:22.127681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.127658    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93a496-57c7cff6f7-7spsm"]
Sep 17 23:37:22.570700 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.570673    2581 scope.go:117] "RemoveContainer" containerID="485582dd520918c3e6943587412d16cf93f4e9fbfab7a328dc15d9bf5983a192"
Sep 17 23:37:22.579119 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:22.579102    2581 scope.go:117] "RemoveContainer" containerID="0e95b7928bdff5f88fa7905642e69cd327f03fdfa1d05ba523a418f3ff60117e"
Sep 17 23:37:23.993737 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:37:23.993706    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfa62470-4985-41b1-b021-17b9e5afd15d" path="/var/lib/kubelet/pods/dfa62470-4985-41b1-b021-17b9e5afd15d/volumes"
Sep 17 23:38:21.275282 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.275243    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"]
Sep 17 23:38:21.275864 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.275506    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd" podUID="d68b5355-4867-481f-b8c3-e274bdbe6d13" containerName="payload-processing" containerID="cri-o://c39b73304807d4f71cfeb21a7b17b50645a64cbbb76fa675fc072aa2ef5039d2" gracePeriod=30
Sep 17 23:38:21.454385 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.454355    2581 generic.go:356] "Generic (PLEG): container finished" podID="d68b5355-4867-481f-b8c3-e274bdbe6d13" containerID="c39b73304807d4f71cfeb21a7b17b50645a64cbbb76fa675fc072aa2ef5039d2" exitCode=0
Sep 17 23:38:21.454574 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.454439    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd" event={"ID":"d68b5355-4867-481f-b8c3-e274bdbe6d13","Type":"ContainerDied","Data":"c39b73304807d4f71cfeb21a7b17b50645a64cbbb76fa675fc072aa2ef5039d2"}
Sep 17 23:38:21.551557 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.551528    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:38:21.611270 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.611236    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68b5355-4867-481f-b8c3-e274bdbe6d13-plugins-config-volume\") pod \"d68b5355-4867-481f-b8c3-e274bdbe6d13\" (UID: \"d68b5355-4867-481f-b8c3-e274bdbe6d13\") "
Sep 17 23:38:21.611455 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.611338    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqdps\" (UniqueName: \"kubernetes.io/projected/d68b5355-4867-481f-b8c3-e274bdbe6d13-kube-api-access-lqdps\") pod \"d68b5355-4867-481f-b8c3-e274bdbe6d13\" (UID: \"d68b5355-4867-481f-b8c3-e274bdbe6d13\") "
Sep 17 23:38:21.611620 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.611592    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d68b5355-4867-481f-b8c3-e274bdbe6d13-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d68b5355-4867-481f-b8c3-e274bdbe6d13" (UID: "d68b5355-4867-481f-b8c3-e274bdbe6d13"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:38:21.611705 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.611688    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d68b5355-4867-481f-b8c3-e274bdbe6d13-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:21.613434 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.613409    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d68b5355-4867-481f-b8c3-e274bdbe6d13-kube-api-access-lqdps" (OuterVolumeSpecName: "kube-api-access-lqdps") pod "d68b5355-4867-481f-b8c3-e274bdbe6d13" (UID: "d68b5355-4867-481f-b8c3-e274bdbe6d13"). InnerVolumeSpecName "kube-api-access-lqdps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:38:21.712572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:21.712535    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqdps\" (UniqueName: \"kubernetes.io/projected/d68b5355-4867-481f-b8c3-e274bdbe6d13-kube-api-access-lqdps\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:22.461260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:22.461224    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd" event={"ID":"d68b5355-4867-481f-b8c3-e274bdbe6d13","Type":"ContainerDied","Data":"b7b29fa851ddd13fec7cf2762c00e4b972ea8b3ec80ce390f08c6b5a3bf7ee57"}
Sep 17 23:38:22.461260 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:22.461251    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"
Sep 17 23:38:22.461862 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:22.461289    2581 scope.go:117] "RemoveContainer" containerID="c39b73304807d4f71cfeb21a7b17b50645a64cbbb76fa675fc072aa2ef5039d2"
Sep 17 23:38:22.480685 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:22.480654    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"]
Sep 17 23:38:22.486789 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:22.486764    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-86dda1-b7b48f5c5-h99sd"]
Sep 17 23:38:23.993372 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:23.993339    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d68b5355-4867-481f-b8c3-e274bdbe6d13" path="/var/lib/kubelet/pods/d68b5355-4867-481f-b8c3-e274bdbe6d13/volumes"
Sep 17 23:38:39.093447 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.093404    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"]
Sep 17 23:38:39.093873 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.093743    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv" podUID="df3b28ed-0290-43bd-a3d2-d4a56bec224d" containerName="payload-pre-processing" containerID="cri-o://93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7" gracePeriod=30
Sep 17 23:38:39.347885 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.347821    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:38:39.392987 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.392957    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df3b28ed-0290-43bd-a3d2-d4a56bec224d-plugins-config-volume\") pod \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\" (UID: \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\") "
Sep 17 23:38:39.392987 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.392991    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bckjg\" (UniqueName: \"kubernetes.io/projected/df3b28ed-0290-43bd-a3d2-d4a56bec224d-kube-api-access-bckjg\") pod \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\" (UID: \"df3b28ed-0290-43bd-a3d2-d4a56bec224d\") "
Sep 17 23:38:39.393424 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.393399    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df3b28ed-0290-43bd-a3d2-d4a56bec224d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df3b28ed-0290-43bd-a3d2-d4a56bec224d" (UID: "df3b28ed-0290-43bd-a3d2-d4a56bec224d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:38:39.395223 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.395196    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df3b28ed-0290-43bd-a3d2-d4a56bec224d-kube-api-access-bckjg" (OuterVolumeSpecName: "kube-api-access-bckjg") pod "df3b28ed-0290-43bd-a3d2-d4a56bec224d" (UID: "df3b28ed-0290-43bd-a3d2-d4a56bec224d"). InnerVolumeSpecName "kube-api-access-bckjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:38:39.494341 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.494300    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df3b28ed-0290-43bd-a3d2-d4a56bec224d-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:39.494341 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.494335    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bckjg\" (UniqueName: \"kubernetes.io/projected/df3b28ed-0290-43bd-a3d2-d4a56bec224d-kube-api-access-bckjg\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:39.560151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.560087    2581 generic.go:356] "Generic (PLEG): container finished" podID="df3b28ed-0290-43bd-a3d2-d4a56bec224d" containerID="93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7" exitCode=0
Sep 17 23:38:39.560151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.560149    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"
Sep 17 23:38:39.560420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.560179    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv" event={"ID":"df3b28ed-0290-43bd-a3d2-d4a56bec224d","Type":"ContainerDied","Data":"93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7"}
Sep 17 23:38:39.560420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.560242    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv" event={"ID":"df3b28ed-0290-43bd-a3d2-d4a56bec224d","Type":"ContainerDied","Data":"d253a2bac1f3df8b84c04252a36b9016234163c62584cdbc21a88b52431f4878"}
Sep 17 23:38:39.560420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.560271    2581 scope.go:117] "RemoveContainer" containerID="93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7"
Sep 17 23:38:39.570819 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.570786    2581 scope.go:117] "RemoveContainer" containerID="93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7"
Sep 17 23:38:39.571222 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:38:39.571191    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7\": container with ID starting with 93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7 not found: ID does not exist" containerID="93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7"
Sep 17 23:38:39.571345 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.571237    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7"} err="failed to get container status \"93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7\": rpc error: code = NotFound desc = could not find container \"93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7\": container with ID starting with 93cd4f0164da45194fcdbc7b57d25e85bc25f38c6eee408e96532af8eea30ec7 not found: ID does not exist"
Sep 17 23:38:39.586694 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.586668    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"]
Sep 17 23:38:39.591634 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.591606    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-e5aae7-7b976f6db-r7lzv"]
Sep 17 23:38:39.994376 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:39.994343    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df3b28ed-0290-43bd-a3d2-d4a56bec224d" path="/var/lib/kubelet/pods/df3b28ed-0290-43bd-a3d2-d4a56bec224d/volumes"
Sep 17 23:38:45.970426 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:45.970392    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"]
Sep 17 23:38:45.970827 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:45.970617    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy" containerID="cri-o://b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b" gracePeriod=30
Sep 17 23:38:51.235032 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.235008    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:38:51.297203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297106    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-socket\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297165    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-istio-token\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297203 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297194    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-data\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297211    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d0b192-a431-443d-a2d2-ca40277114df-istio-podinfo\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297243    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d0b192-a431-443d-a2d2-ca40277114df-istiod-ca-cert\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297285    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-certs\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297366    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-envoy\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297392    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:38:51.297471 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297406    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dctn5\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-kube-api-access-dctn5\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297481    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-credential-socket\") pod \"71d0b192-a431-443d-a2d2-ca40277114df\" (UID: \"71d0b192-a431-443d-a2d2-ca40277114df\") "
Sep 17 23:38:51.297796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297501    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-data" (OuterVolumeSpecName: "istio-data") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:38:51.297796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297633    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:38:51.297796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297672    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71d0b192-a431-443d-a2d2-ca40277114df-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:38:51.298024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297854    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:38:51.298024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297911    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.298024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297930    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-data\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.298024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297939    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d0b192-a431-443d-a2d2-ca40277114df-istiod-ca-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.298024 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.297947    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-workload-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.299712 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.299689    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-kube-api-access-dctn5" (OuterVolumeSpecName: "kube-api-access-dctn5") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "kube-api-access-dctn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:38:51.299712 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.299714    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-istio-token" (OuterVolumeSpecName: "istio-token") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:38:51.299954 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.299938    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71d0b192-a431-443d-a2d2-ca40277114df-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 23:38:51.300000 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.299954    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71d0b192-a431-443d-a2d2-ca40277114df" (UID: "71d0b192-a431-443d-a2d2-ca40277114df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 23:38:51.404953 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.404914    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-istio-envoy\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.404953 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.404954    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dctn5\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-kube-api-access-dctn5\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.405226 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.404976    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d0b192-a431-443d-a2d2-ca40277114df-credential-socket\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.405226 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.404989    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d0b192-a431-443d-a2d2-ca40277114df-istio-token\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.405226 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.405001    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d0b192-a431-443d-a2d2-ca40277114df-istio-podinfo\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:51.637009 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.636973    2581 generic.go:356] "Generic (PLEG): container finished" podID="71d0b192-a431-443d-a2d2-ca40277114df" containerID="b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b" exitCode=0
Sep 17 23:38:51.637220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.637034    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"
Sep 17 23:38:51.637220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.637064    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" event={"ID":"71d0b192-a431-443d-a2d2-ca40277114df","Type":"ContainerDied","Data":"b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b"}
Sep 17 23:38:51.637220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.637171    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr" event={"ID":"71d0b192-a431-443d-a2d2-ca40277114df","Type":"ContainerDied","Data":"add44065967cc32bfe8ecc09bccec9f417078f8c8d8d67344e79e7a26062600e"}
Sep 17 23:38:51.637220 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.637202    2581 scope.go:117] "RemoveContainer" containerID="b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b"
Sep 17 23:38:51.647056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.647039    2581 scope.go:117] "RemoveContainer" containerID="87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c"
Sep 17 23:38:51.654855 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.654837    2581 scope.go:117] "RemoveContainer" containerID="b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b"
Sep 17 23:38:51.655149 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:38:51.655114    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b\": container with ID starting with b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b not found: ID does not exist" containerID="b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b"
Sep 17 23:38:51.655256 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.655153    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b"} err="failed to get container status \"b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b\": rpc error: code = NotFound desc = could not find container \"b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b\": container with ID starting with b8967076a0e2bb4006d746ca32e220df2c6849316e207ecdabacdfb2a600284b not found: ID does not exist"
Sep 17 23:38:51.655256 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.655169    2581 scope.go:117] "RemoveContainer" containerID="87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c"
Sep 17 23:38:51.655438 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:38:51.655424    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c\": container with ID starting with 87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c not found: ID does not exist" containerID="87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c"
Sep 17 23:38:51.655488 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.655441    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c"} err="failed to get container status \"87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c\": rpc error: code = NotFound desc = could not find container \"87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c\": container with ID starting with 87bfae7bdc2db0d6df9939065029bef24d60dc9e7415f111b8c0540597b8555c not found: ID does not exist"
Sep 17 23:38:51.661039 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.661017    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"]
Sep 17 23:38:51.669876 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.669855    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e5aae7-openshift-default-54bff96fdd-fgzgr"]
Sep 17 23:38:51.994692 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:51.994614    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71d0b192-a431-443d-a2d2-ca40277114df" path="/var/lib/kubelet/pods/71d0b192-a431-443d-a2d2-ca40277114df/volumes"
Sep 17 23:38:56.931753 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:56.931717    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"]
Sep 17 23:38:56.932340 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:56.931982    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn" podUID="96c91e78-9d87-4285-aef5-187fdb008c6c" containerName="payload-processing" containerID="cri-o://cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd" gracePeriod=30
Sep 17 23:38:56.940967 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:56.940936    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"]
Sep 17 23:38:56.941212 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:56.941187    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v" podUID="4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c" containerName="payload-pre-processing" containerID="cri-o://bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c" gracePeriod=30
Sep 17 23:38:57.185235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.185177    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:38:57.224245 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.224217    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:38:57.258105 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.258074    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96c91e78-9d87-4285-aef5-187fdb008c6c-plugins-config-volume\") pod \"96c91e78-9d87-4285-aef5-187fdb008c6c\" (UID: \"96c91e78-9d87-4285-aef5-187fdb008c6c\") "
Sep 17 23:38:57.258308 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.258171    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqw9f\" (UniqueName: \"kubernetes.io/projected/96c91e78-9d87-4285-aef5-187fdb008c6c-kube-api-access-wqw9f\") pod \"96c91e78-9d87-4285-aef5-187fdb008c6c\" (UID: \"96c91e78-9d87-4285-aef5-187fdb008c6c\") "
Sep 17 23:38:57.258568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.258534    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c91e78-9d87-4285-aef5-187fdb008c6c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "96c91e78-9d87-4285-aef5-187fdb008c6c" (UID: "96c91e78-9d87-4285-aef5-187fdb008c6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:38:57.260796 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.260748    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c91e78-9d87-4285-aef5-187fdb008c6c-kube-api-access-wqw9f" (OuterVolumeSpecName: "kube-api-access-wqw9f") pod "96c91e78-9d87-4285-aef5-187fdb008c6c" (UID: "96c91e78-9d87-4285-aef5-187fdb008c6c"). InnerVolumeSpecName "kube-api-access-wqw9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:38:57.359332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.359284    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-plugins-config-volume\") pod \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\" (UID: \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\") "
Sep 17 23:38:57.359332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.359336    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5ckp\" (UniqueName: \"kubernetes.io/projected/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-kube-api-access-b5ckp\") pod \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\" (UID: \"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c\") "
Sep 17 23:38:57.359659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.359634    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c" (UID: "4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 23:38:57.359728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.359704    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqw9f\" (UniqueName: \"kubernetes.io/projected/96c91e78-9d87-4285-aef5-187fdb008c6c-kube-api-access-wqw9f\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:57.359728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.359724    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96c91e78-9d87-4285-aef5-187fdb008c6c-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:57.361586 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.361558    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-kube-api-access-b5ckp" (OuterVolumeSpecName: "kube-api-access-b5ckp") pod "4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c" (UID: "4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c"). InnerVolumeSpecName "kube-api-access-b5ckp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:38:57.460819 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.460744    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-plugins-config-volume\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:57.460819 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.460769    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5ckp\" (UniqueName: \"kubernetes.io/projected/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c-kube-api-access-b5ckp\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:38:57.672934 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.672900    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c" containerID="bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c" exitCode=0
Sep 17 23:38:57.673175 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.672941    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"
Sep 17 23:38:57.673175 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.672989    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v" event={"ID":"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c","Type":"ContainerDied","Data":"bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c"}
Sep 17 23:38:57.673175 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.673049    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v" event={"ID":"4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c","Type":"ContainerDied","Data":"a17bcb746dc01b73e4d65211e2a48cd91855fe5443350180531b6f2de44d924f"}
Sep 17 23:38:57.673175 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.673078    2581 scope.go:117] "RemoveContainer" containerID="bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c"
Sep 17 23:38:57.674393 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.674365    2581 generic.go:356] "Generic (PLEG): container finished" podID="96c91e78-9d87-4285-aef5-187fdb008c6c" containerID="cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd" exitCode=0
Sep 17 23:38:57.674530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.674406    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"
Sep 17 23:38:57.674530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.674403    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn" event={"ID":"96c91e78-9d87-4285-aef5-187fdb008c6c","Type":"ContainerDied","Data":"cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd"}
Sep 17 23:38:57.674530 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.674457    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn" event={"ID":"96c91e78-9d87-4285-aef5-187fdb008c6c","Type":"ContainerDied","Data":"9d3940bd6d58a36c194c17ae664af116cf8684f358711de75274088fcf86bd62"}
Sep 17 23:38:57.682692 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.682671    2581 scope.go:117] "RemoveContainer" containerID="bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c"
Sep 17 23:38:57.682984 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:38:57.682959    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c\": container with ID starting with bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c not found: ID does not exist" containerID="bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c"
Sep 17 23:38:57.683051 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.683002    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c"} err="failed to get container status \"bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c\": rpc error: code = NotFound desc = could not find container \"bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c\": container with ID starting with bc955411ebe55ed7d1224002b7ef8651232add2424c1c27df71a0d2165a77f7c not found: ID does not exist"
Sep 17 23:38:57.683051 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.683029    2581 scope.go:117] "RemoveContainer" containerID="cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd"
Sep 17 23:38:57.691253 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.691236    2581 scope.go:117] "RemoveContainer" containerID="cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd"
Sep 17 23:38:57.691539 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:38:57.691521    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd\": container with ID starting with cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd not found: ID does not exist" containerID="cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd"
Sep 17 23:38:57.691600 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.691550    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd"} err="failed to get container status \"cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd\": rpc error: code = NotFound desc = could not find container \"cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd\": container with ID starting with cbfbe2cdffb5560db40f8c31ec5b10d6cb0a9ec6bc376dd0de35d451b9b42ecd not found: ID does not exist"
Sep 17 23:38:57.706568 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.706540    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"]
Sep 17 23:38:57.712684 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.712619    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-6275e9-667f9b6858-5rb2v"]
Sep 17 23:38:57.722780 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.722752    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"]
Sep 17 23:38:57.726922 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.726900    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-6275e9-b4f654fbf-xh9cn"]
Sep 17 23:38:57.994304 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.994227    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c" path="/var/lib/kubelet/pods/4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c/volumes"
Sep 17 23:38:57.994642 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:38:57.994615    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96c91e78-9d87-4285-aef5-187fdb008c6c" path="/var/lib/kubelet/pods/96c91e78-9d87-4285-aef5-187fdb008c6c/volumes"
Sep 17 23:39:06.345235 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.345194    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-565f9646f6-6p9fd"]
Sep 17 23:39:06.345616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.345433    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-565f9646f6-6p9fd" podUID="9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" containerName="manager" containerID="cri-o://a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1" gracePeriod=30
Sep 17 23:39:06.594687 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.594660    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:39:06.736239 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.736207    2581 generic.go:356] "Generic (PLEG): container finished" podID="9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" containerID="a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1" exitCode=0
Sep 17 23:39:06.736442 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.736263    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-6p9fd"
Sep 17 23:39:06.736442 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.736291    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-6p9fd" event={"ID":"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b","Type":"ContainerDied","Data":"a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1"}
Sep 17 23:39:06.736442 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.736363    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-6p9fd" event={"ID":"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b","Type":"ContainerDied","Data":"0c9cc6cb8bb73435607ea8ea0127acd90a2bbbd8df0a58891f666c10c53c2b1f"}
Sep 17 23:39:06.736442 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.736384    2581 scope.go:117] "RemoveContainer" containerID="a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1"
Sep 17 23:39:06.743572 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.743554    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-webhook-cert\") pod \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") "
Sep 17 23:39:06.743693 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.743580    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-metrics-certs\") pod \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") "
Sep 17 23:39:06.743693 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.743657    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r7jm\" (UniqueName: \"kubernetes.io/projected/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-kube-api-access-8r7jm\") pod \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\" (UID: \"9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b\") "
Sep 17 23:39:06.745624 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.745594    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" (UID: "9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:06.745734 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.745684    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-kube-api-access-8r7jm" (OuterVolumeSpecName: "kube-api-access-8r7jm") pod "9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" (UID: "9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b"). InnerVolumeSpecName "kube-api-access-8r7jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:39:06.745734 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.745682    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" (UID: "9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:39:06.746284 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.746269    2581 scope.go:117] "RemoveContainer" containerID="a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1"
Sep 17 23:39:06.746562 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:39:06.746544    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1\": container with ID starting with a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1 not found: ID does not exist" containerID="a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1"
Sep 17 23:39:06.746605 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.746568    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1"} err="failed to get container status \"a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1\": rpc error: code = NotFound desc = could not find container \"a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1\": container with ID starting with a71ed612807c0d5c1a92b288a54a0d806beab05314de1e70ca9e608ad13de3e1 not found: ID does not exist"
Sep 17 23:39:06.844726 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.844676    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-webhook-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:39:06.844726 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.844719    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-metrics-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:39:06.844726 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:06.844733    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r7jm\" (UniqueName: \"kubernetes.io/projected/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b-kube-api-access-8r7jm\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:39:07.061224 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:07.061186    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-565f9646f6-6p9fd"]
Sep 17 23:39:07.067635 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:07.067606    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-565f9646f6-6p9fd"]
Sep 17 23:39:07.994106 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:39:07.994071    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" path="/var/lib/kubelet/pods/9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b/volumes"
Sep 17 23:40:07.870449 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870377    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-565f9646f6-d4zkd"]
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870732    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" containerName="manager"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870743    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" containerName="manager"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870753    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68b5355-4867-481f-b8c3-e274bdbe6d13" containerName="payload-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870758    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68b5355-4867-481f-b8c3-e274bdbe6d13" containerName="payload-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870766    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c" containerName="payload-pre-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870772    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c" containerName="payload-pre-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870780    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df3b28ed-0290-43bd-a3d2-d4a56bec224d" containerName="payload-pre-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870786    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="df3b28ed-0290-43bd-a3d2-d4a56bec224d" containerName="payload-pre-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870792    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c91e78-9d87-4285-aef5-187fdb008c6c" containerName="payload-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870800    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c91e78-9d87-4285-aef5-187fdb008c6c" containerName="payload-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870814    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa62470-4985-41b1-b021-17b9e5afd15d" containerName="payload-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870819    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa62470-4985-41b1-b021-17b9e5afd15d" containerName="payload-processing"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870825    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870830    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870836    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870841    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870847    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8bdd775-6f56-46c8-8a14-104bfe7a33c4" containerName="revoke-keys"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870851    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8bdd775-6f56-46c8-8a14-104bfe7a33c4" containerName="revoke-keys"
Sep 17 23:40:07.870887 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870901    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy"
Sep 17 23:40:07.871629 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870907    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e2b8d78-3630-4894-a3aa-c96d5fcf9e3c" containerName="payload-pre-processing"
Sep 17 23:40:07.871629 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870914    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68b5355-4867-481f-b8c3-e274bdbe6d13" containerName="payload-processing"
Sep 17 23:40:07.871629 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870918    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8bdd775-6f56-46c8-8a14-104bfe7a33c4" containerName="revoke-keys"
Sep 17 23:40:07.871629 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870924    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c06dbcd-7fcd-4c0b-9e7b-9291a09ea72b" containerName="manager"
Sep 17 23:40:07.871629 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870931    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c91e78-9d87-4285-aef5-187fdb008c6c" containerName="payload-processing"
Sep 17 23:40:07.871629 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870937    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfa62470-4985-41b1-b021-17b9e5afd15d" containerName="payload-processing"
Sep 17 23:40:07.871629 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.870943    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="df3b28ed-0290-43bd-a3d2-d4a56bec224d" containerName="payload-pre-processing"
Sep 17 23:40:07.874137 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.874104    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.877871 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.877846    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-snfm4\""
Sep 17 23:40:07.878009 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.877863    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 17 23:40:07.878009 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.877889    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 17 23:40:07.878009 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.877853    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 17 23:40:07.885355 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.885333    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-565f9646f6-d4zkd"]
Sep 17 23:40:07.888818 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.888792    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-webhook-cert\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.888942 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.888851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzvnw\" (UniqueName: \"kubernetes.io/projected/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-kube-api-access-wzvnw\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.888942 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.888910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-metrics-certs\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.989522 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.989489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-webhook-cert\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.989714 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.989540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzvnw\" (UniqueName: \"kubernetes.io/projected/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-kube-api-access-wzvnw\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.989714 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.989703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-metrics-certs\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.992113 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.992089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-metrics-certs\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.992373 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.992354    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-webhook-cert\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:07.997769 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:07.997743    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzvnw\" (UniqueName: \"kubernetes.io/projected/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-kube-api-access-wzvnw\") pod \"maas-controller-565f9646f6-d4zkd\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") " pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:08.185063 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:08.184961    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:08.462032 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:08.462004    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-565f9646f6-d4zkd"]
Sep 17 23:40:08.463441 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:40:08.463412    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f6d2c46_b798_4e4a_b45e_31f69d3ab15e.slice/crio-e6f0d5b5137bc7cfd4a94a79c46519320969ae2a3d3d003b1a7fd69e2c8da56e WatchSource:0}: Error finding container e6f0d5b5137bc7cfd4a94a79c46519320969ae2a3d3d003b1a7fd69e2c8da56e: Status 404 returned error can't find the container with id e6f0d5b5137bc7cfd4a94a79c46519320969ae2a3d3d003b1a7fd69e2c8da56e
Sep 17 23:40:08.464854 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:08.464837    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 23:40:09.067395 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:09.067354    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-d4zkd" event={"ID":"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e","Type":"ContainerStarted","Data":"cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89"}
Sep 17 23:40:09.067752 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:09.067419    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-d4zkd" event={"ID":"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e","Type":"ContainerStarted","Data":"e6f0d5b5137bc7cfd4a94a79c46519320969ae2a3d3d003b1a7fd69e2c8da56e"}
Sep 17 23:40:09.067752 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:09.067539    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:09.086573 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:09.086516    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-565f9646f6-d4zkd" podStartSLOduration=1.620642126 podStartE2EDuration="2.086500372s" totalImagesPullingTime="465.858246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:40:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:40:09.0836309 +0000 UTC m=+2029.087822741" watchObservedRunningTime="2026-09-17 23:40:09.086500372 +0000 UTC m=+2029.090692236"
Sep 17 23:40:20.073873 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:20.073842    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:27.275131 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.275090    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-565f9646f6-d4zkd"]
Sep 17 23:40:27.275630 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.275293    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-565f9646f6-d4zkd" podUID="8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" containerName="manager" containerID="cri-o://cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89" gracePeriod=30
Sep 17 23:40:27.519176 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.519147    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:27.564729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.564660    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzvnw\" (UniqueName: \"kubernetes.io/projected/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-kube-api-access-wzvnw\") pod \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") "
Sep 17 23:40:27.564729 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.564728    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-webhook-cert\") pod \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") "
Sep 17 23:40:27.564945 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.564765    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-metrics-certs\") pod \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\" (UID: \"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e\") "
Sep 17 23:40:27.567067 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.567036    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" (UID: "8f6d2c46-b798-4e4a-b45e-31f69d3ab15e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:40:27.567301 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.567281    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" (UID: "8f6d2c46-b798-4e4a-b45e-31f69d3ab15e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 23:40:27.567394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.567378    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-kube-api-access-wzvnw" (OuterVolumeSpecName: "kube-api-access-wzvnw") pod "8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" (UID: "8f6d2c46-b798-4e4a-b45e-31f69d3ab15e"). InnerVolumeSpecName "kube-api-access-wzvnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 23:40:27.665596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.665555    2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-webhook-cert\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:40:27.665596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.665588    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-metrics-certs\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:40:27.665596 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:27.665602    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzvnw\" (UniqueName: \"kubernetes.io/projected/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e-kube-api-access-wzvnw\") on node \"ip-10-0-128-233.ec2.internal\" DevicePath \"\""
Sep 17 23:40:28.171489 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.171456    2581 generic.go:356] "Generic (PLEG): container finished" podID="8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" containerID="cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89" exitCode=0
Sep 17 23:40:28.171671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.171521    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-d4zkd"
Sep 17 23:40:28.171671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.171544    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-d4zkd" event={"ID":"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e","Type":"ContainerDied","Data":"cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89"}
Sep 17 23:40:28.171671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.171594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-d4zkd" event={"ID":"8f6d2c46-b798-4e4a-b45e-31f69d3ab15e","Type":"ContainerDied","Data":"e6f0d5b5137bc7cfd4a94a79c46519320969ae2a3d3d003b1a7fd69e2c8da56e"}
Sep 17 23:40:28.171671 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.171616    2581 scope.go:117] "RemoveContainer" containerID="cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89"
Sep 17 23:40:28.183177 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.183156    2581 scope.go:117] "RemoveContainer" containerID="cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89"
Sep 17 23:40:28.183632 ip-10-0-128-233 kubenswrapper[2581]: E0917 23:40:28.183606    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89\": container with ID starting with cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89 not found: ID does not exist" containerID="cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89"
Sep 17 23:40:28.183719 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.183644    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89"} err="failed to get container status \"cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89\": rpc error: code = NotFound desc = could not find container \"cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89\": container with ID starting with cf1e6a131ce543dcfcc16eed0fb43b6f1331bc20a4bd006e7d140006a6c95c89 not found: ID does not exist"
Sep 17 23:40:28.189569 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.189546    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-565f9646f6-d4zkd"]
Sep 17 23:40:28.193221 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.193198    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-565f9646f6-d4zkd"]
Sep 17 23:40:28.968230 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.968197    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-565f9646f6-jxd68"]
Sep 17 23:40:28.968591 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.968563    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" containerName="manager"
Sep 17 23:40:28.968591 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.968574    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" containerName="manager"
Sep 17 23:40:28.968678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.968634    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" containerName="manager"
Sep 17 23:40:28.968678 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.968643    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d0b192-a431-443d-a2d2-ca40277114df" containerName="istio-proxy"
Sep 17 23:40:28.973040 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.973022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:28.977117 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.977089    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 17 23:40:28.977252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.977167    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-snfm4\""
Sep 17 23:40:28.977252 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.977187    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 17 23:40:28.977395 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.977291    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 17 23:40:28.983215 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:28.983195    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-565f9646f6-jxd68"]
Sep 17 23:40:29.078906 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.078875    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-webhook-cert\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.079077 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.078931    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzr6z\" (UniqueName: \"kubernetes.io/projected/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-kube-api-access-tzr6z\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.079077 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.079030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-metrics-certs\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.179458 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.179427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-webhook-cert\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.179616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.179481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr6z\" (UniqueName: \"kubernetes.io/projected/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-kube-api-access-tzr6z\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.179616 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.179511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-metrics-certs\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.181945 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.181914    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-webhook-cert\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.182053 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.181955    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-metrics-certs\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.187583 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.187562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzr6z\" (UniqueName: \"kubernetes.io/projected/f6dcb768-6b7e-4b76-ae30-b1e55081bd26-kube-api-access-tzr6z\") pod \"maas-controller-565f9646f6-jxd68\" (UID: \"f6dcb768-6b7e-4b76-ae30-b1e55081bd26\") " pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.283648 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.283566    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:29.556077 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.556054    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-565f9646f6-jxd68"]
Sep 17 23:40:29.558006 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:40:29.557970    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6dcb768_6b7e_4b76_ae30_b1e55081bd26.slice/crio-bc331538aa68913740e24398eb1f45e5599e3ee71d69e7489fcfd386634b3623 WatchSource:0}: Error finding container bc331538aa68913740e24398eb1f45e5599e3ee71d69e7489fcfd386634b3623: Status 404 returned error can't find the container with id bc331538aa68913740e24398eb1f45e5599e3ee71d69e7489fcfd386634b3623
Sep 17 23:40:29.995038 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:29.995004    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f6d2c46-b798-4e4a-b45e-31f69d3ab15e" path="/var/lib/kubelet/pods/8f6d2c46-b798-4e4a-b45e-31f69d3ab15e/volumes"
Sep 17 23:40:30.183845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:30.183757    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-jxd68" event={"ID":"f6dcb768-6b7e-4b76-ae30-b1e55081bd26","Type":"ContainerStarted","Data":"fcf49865fcaf65bc93acf2b8d92a50104c66492e2dcc393420d983ea0aa29bb8"}
Sep 17 23:40:30.183845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:30.183813    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-565f9646f6-jxd68" event={"ID":"f6dcb768-6b7e-4b76-ae30-b1e55081bd26","Type":"ContainerStarted","Data":"bc331538aa68913740e24398eb1f45e5599e3ee71d69e7489fcfd386634b3623"}
Sep 17 23:40:30.184032 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:30.183952    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:40:30.213541 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:30.213483    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-565f9646f6-jxd68" podStartSLOduration=1.911593312 podStartE2EDuration="2.213465918s" totalImagesPullingTime="301.872606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 23:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:40:30.209304252 +0000 UTC m=+2050.213496091" watchObservedRunningTime="2026-09-17 23:40:30.213465918 +0000 UTC m=+2050.217657765"
Sep 17 23:40:41.191781 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:40:41.191704    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-565f9646f6-jxd68"
Sep 17 23:43:22.692325 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:43:22.692289    2581 scope.go:117] "RemoveContainer" containerID="41baecea5777c5132d55557adb89df7a88e98d7d4f05ae8b406020ce41c2888f"
Sep 17 23:44:54.149982 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:44:54.149943    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b4c854f84-t9xxj_f9cdf16e-cddb-417d-a457-cc347a236cbd/authorino/0.log"
Sep 17 23:44:59.734659 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:44:59.734620    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-565f9646f6-jxd68_f6dcb768-6b7e-4b76-ae30-b1e55081bd26/manager/0.log"
Sep 17 23:44:59.840938 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:44:59.840905    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-v5lxw_42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1/server/0.log"
Sep 17 23:44:59.965411 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:44:59.965368    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r48zj_359bfa42-ebaf-4f98-b787-6e024edd5b4b/manager/0.log"
Sep 17 23:45:00.134496 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.134456    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp"]
Sep 17 23:45:00.137974 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.137951    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp"
Sep 17 23:45:00.140846 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.140825    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-s5ldj\""
Sep 17 23:45:00.147238 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.147209    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp"]
Sep 17 23:45:00.149043 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.149025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp"
Sep 17 23:45:00.317207 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.317181    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d5976b594-lltk9_237e6004-a756-4f77-b9b4-783adee35380/copy-manifests/0.log"
Sep 17 23:45:00.324853 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.324830    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d5976b594-lltk9_237e6004-a756-4f77-b9b4-783adee35380/manager/0.log"
Sep 17 23:45:00.427394 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.427367    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp"]
Sep 17 23:45:00.428435 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:45:00.428411    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod240ab8c4_86f2_4772_95ec_293eb4c97255.slice/crio-1d89022e58f8c7dc393e61b9972bcab9609214ac166a2f29d9accb6bceda2a58 WatchSource:0}: Error finding container 1d89022e58f8c7dc393e61b9972bcab9609214ac166a2f29d9accb6bceda2a58: Status 404 returned error can't find the container with id 1d89022e58f8c7dc393e61b9972bcab9609214ac166a2f29d9accb6bceda2a58
Sep 17 23:45:00.649533 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.649500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp" event={"ID":"240ab8c4-86f2-4772-95ec-293eb4c97255","Type":"ContainerStarted","Data":"1d89022e58f8c7dc393e61b9972bcab9609214ac166a2f29d9accb6bceda2a58"}
Sep 17 23:45:00.905539 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:00.905509    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29828130-zgdhk_2736b23a-1440-4d3f-b591-a519b8842554/cleanup/0.log"
Sep 17 23:45:01.656305 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:01.656207    2581 generic.go:356] "Generic (PLEG): container finished" podID="240ab8c4-86f2-4772-95ec-293eb4c97255" containerID="101c7c6ca238d7fd9f56f23c4695526189989d7f1375d7590b494e2f2b7a847a" exitCode=0
Sep 17 23:45:01.656467 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:01.656300    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp" event={"ID":"240ab8c4-86f2-4772-95ec-293eb4c97255","Type":"ContainerDied","Data":"101c7c6ca238d7fd9f56f23c4695526189989d7f1375d7590b494e2f2b7a847a"}
Sep 17 23:45:01.884745 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:01.884715    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b4c854f84-t9xxj_f9cdf16e-cddb-417d-a457-cc347a236cbd/authorino/0.log"
Sep 17 23:45:02.792109 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:02.792083    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp"
Sep 17 23:45:02.918309 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:02.918227    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq_3336558e-734f-49fb-b82a-0e05a2a2a582/istio-proxy/0.log"
Sep 17 23:45:03.046056 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:03.046024    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-crrwm_7732a4eb-adc8-4011-af97-586a4bf96319/discovery/0.log"
Sep 17 23:45:03.379153 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:03.379110    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-k5pcr_615d0917-5b05-44ad-856a-f7e723ba954d/istio-proxy/1.log"
Sep 17 23:45:03.611676 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:03.611649    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-kd6gq_9babfa47-30fb-4a73-88f9-8a090d496b96/payload-processing/0.log"
Sep 17 23:45:03.668282 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:03.668205    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp"
Sep 17 23:45:03.668282 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:03.668221    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29828145-f6dpp" event={"ID":"240ab8c4-86f2-4772-95ec-293eb4c97255","Type":"ContainerDied","Data":"1d89022e58f8c7dc393e61b9972bcab9609214ac166a2f29d9accb6bceda2a58"}
Sep 17 23:45:03.668282 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:03.668260    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d89022e58f8c7dc393e61b9972bcab9609214ac166a2f29d9accb6bceda2a58"
Sep 17 23:45:04.159688 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:04.159646    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jzh59_b051d47e-06a6-47fe-9ffd-83dcffbec306/main/0.log"
Sep 17 23:45:04.488415 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:04.488332    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fl7rr9_832b8fb5-86ff-4055-8537-a69fac42b24a/main/0.log"
Sep 17 23:45:12.855779 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:12.855744    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-csnkx_50b8ad4b-7ba8-4b82-9a42-6acdf46910b8/global-pull-secret-syncer/0.log"
Sep 17 23:45:12.989272 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:12.989238    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jq5sx_89970b05-164f-41fc-b269-0ae42eeddb40/konnectivity-agent/0.log"
Sep 17 23:45:13.042405 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:13.042372    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-233.ec2.internal_e6989cb969ca76f3d903d482174d8453/haproxy/0.log"
Sep 17 23:45:17.547087 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:17.547051    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6b4c854f84-t9xxj_f9cdf16e-cddb-417d-a457-cc347a236cbd/authorino/0.log"
Sep 17 23:45:19.102977 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.102950    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9ff4ec7-81a4-4e14-87b0-01f028df392b/alertmanager/0.log"
Sep 17 23:45:19.122899 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.122871    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9ff4ec7-81a4-4e14-87b0-01f028df392b/config-reloader/0.log"
Sep 17 23:45:19.146258 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.146231    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9ff4ec7-81a4-4e14-87b0-01f028df392b/kube-rbac-proxy-web/0.log"
Sep 17 23:45:19.165662 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.165642    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9ff4ec7-81a4-4e14-87b0-01f028df392b/kube-rbac-proxy/0.log"
Sep 17 23:45:19.189675 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.189653    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9ff4ec7-81a4-4e14-87b0-01f028df392b/kube-rbac-proxy-metric/0.log"
Sep 17 23:45:19.210165 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.210146    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9ff4ec7-81a4-4e14-87b0-01f028df392b/prom-label-proxy/0.log"
Sep 17 23:45:19.229993 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.229969    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9ff4ec7-81a4-4e14-87b0-01f028df392b/init-config-reloader/0.log"
Sep 17 23:45:19.442006 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.441976    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7ttns_b313c49a-3247-4725-ba81-5e936726b059/node-exporter/0.log"
Sep 17 23:45:19.466172 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.466147    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7ttns_b313c49a-3247-4725-ba81-5e936726b059/kube-rbac-proxy/0.log"
Sep 17 23:45:19.489829 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:19.489810    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7ttns_b313c49a-3247-4725-ba81-5e936726b059/init-textfile/0.log"
Sep 17 23:45:20.076221 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.076144    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-545f84c569-5cz58_c00e524b-5b91-40e0-ba70-5a59b1d9f96d/telemeter-client/0.log"
Sep 17 23:45:20.097271 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.097249    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-545f84c569-5cz58_c00e524b-5b91-40e0-ba70-5a59b1d9f96d/reload/0.log"
Sep 17 23:45:20.118149 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.118102    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-545f84c569-5cz58_c00e524b-5b91-40e0-ba70-5a59b1d9f96d/kube-rbac-proxy/0.log"
Sep 17 23:45:20.159590 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.159563    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d899f7d48-96n75_abcd80ee-e836-4577-92f4-97722f9b41b1/thanos-query/0.log"
Sep 17 23:45:20.189726 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.189702    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d899f7d48-96n75_abcd80ee-e836-4577-92f4-97722f9b41b1/kube-rbac-proxy-web/0.log"
Sep 17 23:45:20.210498 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.210476    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d899f7d48-96n75_abcd80ee-e836-4577-92f4-97722f9b41b1/kube-rbac-proxy/0.log"
Sep 17 23:45:20.238595 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.238572    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d899f7d48-96n75_abcd80ee-e836-4577-92f4-97722f9b41b1/prom-label-proxy/0.log"
Sep 17 23:45:20.264541 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.264522    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d899f7d48-96n75_abcd80ee-e836-4577-92f4-97722f9b41b1/kube-rbac-proxy-rules/0.log"
Sep 17 23:45:20.283516 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:20.283492    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d899f7d48-96n75_abcd80ee-e836-4577-92f4-97722f9b41b1/kube-rbac-proxy-metrics/0.log"
Sep 17 23:45:22.826546 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.826513    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"]
Sep 17 23:45:22.826892 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.826885    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="240ab8c4-86f2-4772-95ec-293eb4c97255" containerName="cleanup"
Sep 17 23:45:22.826946 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.826895    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="240ab8c4-86f2-4772-95ec-293eb4c97255" containerName="cleanup"
Sep 17 23:45:22.826982 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.826962    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="240ab8c4-86f2-4772-95ec-293eb4c97255" containerName="cleanup"
Sep 17 23:45:22.830003 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.829984    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:22.832728 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.832703    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xj26g\"/\"openshift-service-ca.crt\""
Sep 17 23:45:22.832845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.832738    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xj26g\"/\"default-dockercfg-qmmhv\""
Sep 17 23:45:22.832845 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.832823    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xj26g\"/\"kube-root-ca.crt\""
Sep 17 23:45:22.838205 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:22.838184    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"]
Sep 17 23:45:23.011327 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.011292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjmhj\" (UniqueName: \"kubernetes.io/projected/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-kube-api-access-cjmhj\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.011491 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.011355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-podres\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.011491 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.011415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-proc\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.011491 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.011469    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-sys\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.011607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.011498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-lib-modules\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.112949 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.112922    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-podres\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.113099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.112958    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-proc\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.113099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.113013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-sys\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.113099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.113043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-lib-modules\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.113099 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.113090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-podres\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.113343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.113101    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-proc\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.113343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.113145    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-sys\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.113343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.113162    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-lib-modules\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.113343 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.113179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjmhj\" (UniqueName: \"kubernetes.io/projected/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-kube-api-access-cjmhj\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.121395 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.121369    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjmhj\" (UniqueName: \"kubernetes.io/projected/67c027d5-a3be-4abc-9f0c-6d9f1af113ae-kube-api-access-cjmhj\") pod \"perf-node-gather-daemonset-724vm\" (UID: \"67c027d5-a3be-4abc-9f0c-6d9f1af113ae\") " pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.140831 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.140810    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.411279 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.411251    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"]
Sep 17 23:45:23.412553 ip-10-0-128-233 kubenswrapper[2581]: W0917 23:45:23.412526    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod67c027d5_a3be_4abc_9f0c_6d9f1af113ae.slice/crio-39d609c3792e6b065f732e45ad992ff6c2966f86bc35004df4e3cfac2651a89e WatchSource:0}: Error finding container 39d609c3792e6b065f732e45ad992ff6c2966f86bc35004df4e3cfac2651a89e: Status 404 returned error can't find the container with id 39d609c3792e6b065f732e45ad992ff6c2966f86bc35004df4e3cfac2651a89e
Sep 17 23:45:23.776288 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.776184    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm" event={"ID":"67c027d5-a3be-4abc-9f0c-6d9f1af113ae","Type":"ContainerStarted","Data":"64ec62a6a05d5403d75790f6a7e0a53833fb0d4f5bfee7c32a13a01bff574337"}
Sep 17 23:45:23.776288 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.776246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm" event={"ID":"67c027d5-a3be-4abc-9f0c-6d9f1af113ae","Type":"ContainerStarted","Data":"39d609c3792e6b065f732e45ad992ff6c2966f86bc35004df4e3cfac2651a89e"}
Sep 17 23:45:23.776288 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.776276    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:23.793341 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:23.793290    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm" podStartSLOduration=1.793272916 podStartE2EDuration="1.793272916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 23:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 23:45:23.791561656 +0000 UTC m=+2343.795753496" watchObservedRunningTime="2026-09-17 23:45:23.793272916 +0000 UTC m=+2343.797464756"
Sep 17 23:45:24.765085 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:24.765058    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9sscn_227b9f31-6478-4430-ae85-fa6085c296cd/dns/0.log"
Sep 17 23:45:24.784623 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:24.784597    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9sscn_227b9f31-6478-4430-ae85-fa6085c296cd/kube-rbac-proxy/0.log"
Sep 17 23:45:24.895425 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:24.895396    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-46t4x_1695e8f9-ec8b-41db-a230-369fdae10dcb/dns-node-resolver/0.log"
Sep 17 23:45:25.480746 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:25.480713    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7646669d8b-fwlsv_fd708c1e-4ada-427c-9d9e-fd47a0980689/registry/0.log"
Sep 17 23:45:25.503163 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:25.503107    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7287k_7cd72f17-5179-44c0-a06f-d330a2766e78/node-ca/0.log"
Sep 17 23:45:26.472354 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:26.472313    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq_3336558e-734f-49fb-b82a-0e05a2a2a582/istio-proxy/0.log"
Sep 17 23:45:27.239681 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:27.239650    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-crrwm_7732a4eb-adc8-4011-af97-586a4bf96319/discovery/0.log"
Sep 17 23:45:27.315361 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:27.315335    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-k5pcr_615d0917-5b05-44ad-856a-f7e723ba954d/istio-proxy/0.log"
Sep 17 23:45:27.320420 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:27.320394    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-k5pcr_615d0917-5b05-44ad-856a-f7e723ba954d/istio-proxy/1.log"
Sep 17 23:45:27.405536 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:27.405505    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-kd6gq_9babfa47-30fb-4a73-88f9-8a090d496b96/payload-processing/0.log"
Sep 17 23:45:28.344953 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:28.344913    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x7bqz_9bdaf271-bcf3-4c34-865f-4ec53f8d5c37/serve-healthcheck-canary/0.log"
Sep 17 23:45:28.843815 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:28.843784    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tk7qr_ac663cea-8971-4476-b0ef-fc259d52500f/kube-rbac-proxy/0.log"
Sep 17 23:45:28.862370 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:28.862342    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tk7qr_ac663cea-8971-4476-b0ef-fc259d52500f/exporter/0.log"
Sep 17 23:45:28.884454 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:28.884427    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tk7qr_ac663cea-8971-4476-b0ef-fc259d52500f/extractor/0.log"
Sep 17 23:45:29.792033 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:29.792006    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xj26g/perf-node-gather-daemonset-724vm"
Sep 17 23:45:31.528876 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:31.528846    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-565f9646f6-jxd68_f6dcb768-6b7e-4b76-ae30-b1e55081bd26/manager/0.log"
Sep 17 23:45:31.549892 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:31.549857    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-v5lxw_42a82f45-f8f2-4ffd-baf8-e4ec2d4564a1/server/0.log"
Sep 17 23:45:31.687627 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:31.687597    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-r48zj_359bfa42-ebaf-4f98-b787-6e024edd5b4b/manager/0.log"
Sep 17 23:45:31.859300 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:31.859268    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d5976b594-lltk9_237e6004-a756-4f77-b9b4-783adee35380/manager/0.log"
Sep 17 23:45:31.877029 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:31.877005    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d5976b594-lltk9_237e6004-a756-4f77-b9b4-783adee35380/copy-manifests/0.log"
Sep 17 23:45:32.636498 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:32.636462    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq_3336558e-734f-49fb-b82a-0e05a2a2a582/istio-proxy/0.log"
Sep 17 23:45:33.155614 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:33.155570    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-ccddb67c-f8ggm_bb6ce8a8-3e24-4c1d-aba1-b974b2cc0677/manager/0.log"
Sep 17 23:45:33.442225 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:33.442114    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-crrwm_7732a4eb-adc8-4011-af97-586a4bf96319/discovery/0.log"
Sep 17 23:45:33.495735 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:33.495703    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-k5pcr_615d0917-5b05-44ad-856a-f7e723ba954d/istio-proxy/0.log"
Sep 17 23:45:33.500951 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:33.500923    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-k5pcr_615d0917-5b05-44ad-856a-f7e723ba954d/istio-proxy/1.log"
Sep 17 23:45:33.559151 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:33.559093    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-kd6gq_9babfa47-30fb-4a73-88f9-8a090d496b96/payload-processing/0.log"
Sep 17 23:45:35.090310 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:35.090278    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-nkdvq_3336558e-734f-49fb-b82a-0e05a2a2a582/istio-proxy/0.log"
Sep 17 23:45:35.899384 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:35.899302    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-crrwm_7732a4eb-adc8-4011-af97-586a4bf96319/discovery/0.log"
Sep 17 23:45:35.961021 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:35.960992    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-k5pcr_615d0917-5b05-44ad-856a-f7e723ba954d/istio-proxy/0.log"
Sep 17 23:45:35.966352 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:35.966322    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-k5pcr_615d0917-5b05-44ad-856a-f7e723ba954d/istio-proxy/1.log"
Sep 17 23:45:36.033146 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:36.033101    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-kd6gq_9babfa47-30fb-4a73-88f9-8a090d496b96/payload-processing/0.log"
Sep 17 23:45:39.661198 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:39.661170    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbnmm_1049d3e7-bedd-4ac4-b35d-f156a480d857/kube-multus-additional-cni-plugins/0.log"
Sep 17 23:45:39.684535 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:39.684510    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbnmm_1049d3e7-bedd-4ac4-b35d-f156a480d857/egress-router-binary-copy/0.log"
Sep 17 23:45:39.705607 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:39.705578    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbnmm_1049d3e7-bedd-4ac4-b35d-f156a480d857/cni-plugins/0.log"
Sep 17 23:45:39.730723 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:39.730699    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbnmm_1049d3e7-bedd-4ac4-b35d-f156a480d857/bond-cni-plugin/0.log"
Sep 17 23:45:39.752814 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:39.752790    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbnmm_1049d3e7-bedd-4ac4-b35d-f156a480d857/routeoverride-cni/0.log"
Sep 17 23:45:39.772912 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:39.772891    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbnmm_1049d3e7-bedd-4ac4-b35d-f156a480d857/whereabouts-cni-bincopy/0.log"
Sep 17 23:45:39.796332 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:39.796314    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dbnmm_1049d3e7-bedd-4ac4-b35d-f156a480d857/whereabouts-cni/0.log"
Sep 17 23:45:40.174369 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:40.174334    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ntq4l_e8b6666b-4910-4bd4-a3a3-f1a086a67052/kube-multus/0.log"
Sep 17 23:45:40.245167 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:40.245120    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-khgbr_9f1c5761-f83c-4f1c-8994-8de155256444/network-metrics-daemon/0.log"
Sep 17 23:45:40.259684 ip-10-0-128-233 kubenswrapper[2581]: I0917 23:45:40.259657    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-khgbr_9f1c5761-f83c-4f1c-8994-8de155256444/kube-rbac-proxy/0.log"